Concurrent tools: schedule the dependency graph, not the visible list
Separate data dependencies, resource conflicts, deterministic joins, structured child ownership, and bounded parallel execution.
On this page
The question for today
What you’ll understand
- Construct a tool-call dependency graph with resource constraints.
- Calculate ideal work and critical-path bounds.
- Preserve accepted result identity and child ownership under parallel completion.
Parallel tool calls can reduce latency when they are independent, but a list of model-suggested actions is not automatically a safe parallel batch. Some calls consume another call's artifact, mutate shared state, or compete for one approval or budget. Build a dependency and resource model, then define how results, failures, and cancellations join back into the durable run.
Independence includes data and effects
Add a dependency edge when a call needs an earlier result, resource version, approval, or committed artifact. Calls that write shared state may need an order even when their argument objects are disjoint. Read-only calls can still require a common snapshot for a meaningful combined answer. Treat shared quotas, provider rate limits, and authorization scope as scheduling constraints. An external service's actual consistency and conditional-write contract determine which apparent operations commute.
Before dispatch, assign stable logical step and effect identities and reserve the resources required by the chosen concurrency policy. A late completion from a replaced worker must not attach to the wrong branch. Preserve output-to-input lineage so the join uses the intended artifact versions. Parallelism should not make a dependent call guess a missing value from a stale cache or model memory. If one branch fails, decide whether independent successful work remains useful or must be compensated under the declared workflow policy.
Latency bounds require matching scheduling assumptions
For fixed task durations, total work is the sum of durations and the dependency critical path is the longest required chain. With P identical worker slots, work divided by P and the critical path are lower bounds on makespan under an idealized schedule. They do not promise that a real queue, network, or external API reaches the bound. Startup, contention, batching, retries, and resource-specific limits can dominate. More concurrency can increase exposed queueing or exhaust a provider quota rather than reduce latency.
Structured concurrency gives each spawned task a scope whose owner waits for or handles its completion. Python 3.14 TaskGroup is one concrete implementation with documented failure and cancellation behavior. It does not make remote side effects transactional or guarantee a child thread or server instantly stops. A durable agent also needs persistent branch identities and joins across worker crashes. Record the accepted completion order or use a deterministic identity-based join when order matters; arrival order is not automatically a stable planning input.
Test concurrency policies with adversarial completion order
Run the same logical graph with reversed completion order, one delayed branch, duplicate results, a failed prerequisite, and cancellation during a join. Verify no dependent effect starts before its required outcome is accepted, and no result is paired with another branch's arguments. Test shared-budget reservations atomically. Compare serial and bounded-concurrent variants with the same effects and data snapshot. A latency improvement obtained by omitting a required dependency or dropping failed branches is not equivalent execution.
The queue lab isolates one capacity mismatch. Reset, set λ=4 and μ=3, predict q(2)=6, then use four Steps. Separately Reset, set λ=4 and μ=5, predict q(2)=2, then use four Steps. Both start with four abstract work units. The instrument has no dependency graph, worker count, or tool service, and its separate residence-time model has additional assumptions. Use it to see why offered work and service must be measured together, then test the actual scheduler and provider constraints.
Work it through
Schedule a small dependency graph
Calls A and B take four and six seconds and can start together. C takes three seconds after both finish. D takes two seconds after A only. Use two ideal slots with no overhead or other resource limits.
At time zero, start A and B. A ends at four; B ends at six.
Run D from four to six in A's released slot. At six, both A and B are complete, so C runs until nine.
Makespan is nine seconds versus 4+6+3+2=15 seconds for a fully serial order. The critical chain B→C is nine, and total work divided by two is 7.5, so the ideal bound is reached under these assumptions.
The safe schedule exploits actual independence while respecting both prerequisites of C. It is a graph-specific result, not a blanket speedup from parallel calls.
The living notebook
Explore the model
Interactive lab · queue
Concurrency still faces a service constraint
Reset, set λ=4 and μ=3, predict q(2)=6, then use four Steps. Separately Reset, set λ=4 and μ=5, predict q(2)=2, then use four Steps. This initial-four fluid queue has no dependency edges, worker slots, approvals, or provider API.
- Fluid backlog
- 4
- Net rate · λ − μ
- −1 /s
- M/M/1 mean residence
- 1 s
Fluid backlog starts at 4 and follows max(0, 4 + (λ − μ)t). The separate M/M/1 residence-time formula assumes Poisson arrivals, exponential service and λ < μ. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.
Look forBounded parallelism must preserve dependencies and match actual shared-resource capacity.
Pause & reason
Check your understanding
Your turn
Try it yourself
Independent A and B take two and five seconds. C takes four seconds and requires both. With two ideal slots and no overhead, find makespan and the fully serial duration.
Take a moment to make a prediction or work it out on paper. Open a hint when you need one.
Give me a hint
C starts after the slower prerequisite finishes.
Compare with the solution
A and B finish by time five, then C finishes at nine. Fully serial work takes 2+5+4=11 seconds. The nine-second dependency chain B→C is the relevant ideal lower bound; real tool overhead or resource limits can add time.
Keep the idea
Take it with you
- Tool independence includes versions, effects, scopes, and shared resources.
- Critical-path bounds need explicit scheduling assumptions.
- Durable joins and structured ownership remain necessary under parallel completion.
Read further
Original sources and deeper explanations to follow at your own pace.
Optional progress, saved in this browser when available. Revisit any lesson at any time.
Lesson not marked complete.