research28 min studyLesson 887 in Engineering

Concurrent tools: schedule the dependency graph, not the visible list

Separate data dependencies, resource conflicts, deterministic joins, structured child ownership, and bounded parallel execution.

On this page

What you’ll understand

  • Construct a tool-call dependency graph with resource constraints.
  • Calculate ideal work and critical-path bounds.
  • Preserve accepted result identity and child ownership under parallel completion.

Parallel tool calls can reduce latency when they are independent, but a list of model-suggested actions is not automatically a safe parallel batch. Some calls consume another call's artifact, mutate shared state, or compete for one approval or budget. Build a dependency and resource model, then define how results, failures, and cancellations join back into the durable run.

Independence includes data and effects

Add a dependency edge when a call needs an earlier result, resource version, approval, or committed artifact. Calls that write shared state may need an order even when their argument objects are disjoint. Read-only calls can still require a common snapshot for a meaningful combined answer. Treat shared quotas, provider rate limits, and authorization scope as scheduling constraints. An external service's actual consistency and conditional-write contract determine which apparent operations commute.

Before dispatch, assign stable logical step and effect identities and reserve the resources required by the chosen concurrency policy. A late completion from a replaced worker must not attach to the wrong branch. Preserve output-to-input lineage so the join uses the intended artifact versions. Parallelism should not make a dependent call guess a missing value from a stale cache or model memory. If one branch fails, decide whether independent successful work remains useful or must be compensated under the declared workflow policy.

u≺v⟹Ready⁡(v)⇒RequiredOutcome⁡(u) is durably availableu\prec v\Longrightarrow\operatorname{Ready}(v)\Rightarrow\operatorname{RequiredOutcome}(u)\ \text{is durably available}
The proposed dependency relation includes the outcome required by v, which may be a confirmed result, an approved artifact, or a declared failure branch. It is not enough that u was merely dispatched. Cycles require another explicit protocol rather than a naive DAG scheduler.↔ Scroll wide equations horizontally.

Latency bounds require matching scheduling assumptions

For fixed task durations, total work is the sum of durations and the dependency critical path is the longest required chain. With P identical worker slots, work divided by P and the critical path are lower bounds on makespan under an idealized schedule. They do not promise that a real queue, network, or external API reaches the bound. Startup, contention, batching, retries, and resource-specific limits can dominate. More concurrency can increase exposed queueing or exhaust a provider quota rather than reduce latency.

Structured concurrency gives each spawned task a scope whose owner waits for or handles its completion. Python 3.14 TaskGroup is one concrete implementation with documented failure and cancellation behavior. It does not make remote side effects transactional or guarantee a child thread or server instantly stops. A durable agent also needs persistent branch identities and joins across worker crashes. Record the accepted completion order or use a deterministic identity-based join when order matters; arrival order is not automatically a stable planning input.

Tideal≥max⁡ ⁣(Lcritical,∑idiP)T_{\rm ideal}\geq\max\!\left(L_{\rm critical},\frac{\sum_i d_i}{P}\right)
d_i are fixed nonnegative task durations and P≥1 identical ideal slots. L_critical is the longest dependency-chain duration. The lower bound omits resource heterogeneity and overhead, and is not an achieved latency guarantee for a tool service.↔ Scroll wide equations horizontally.

Test concurrency policies with adversarial completion order

Run the same logical graph with reversed completion order, one delayed branch, duplicate results, a failed prerequisite, and cancellation during a join. Verify no dependent effect starts before its required outcome is accepted, and no result is paired with another branch's arguments. Test shared-budget reservations atomically. Compare serial and bounded-concurrent variants with the same effects and data snapshot. A latency improvement obtained by omitting a required dependency or dropping failed branches is not equivalent execution.

The queue lab isolates one capacity mismatch. Reset, set λ=4 and μ=3, predict q(2)=6, then use four Steps. Separately Reset, set λ=4 and μ=5, predict q(2)=2, then use four Steps. Both start with four abstract work units. The instrument has no dependency graph, worker count, or tool service, and its separate residence-time model has additional assumptions. Use it to see why offered work and service must be measured together, then test the actual scheduler and provider constraints.

Schedule a small dependency graph

Calls A and B take four and six seconds and can start together. C takes three seconds after both finish. D takes two seconds after A only. Use two ideal slots with no overhead or other resource limits.

  1. At time zero, start A and B. A ends at four; B ends at six.

  2. Run D from four to six in A's released slot. At six, both A and B are complete, so C runs until nine.

  3. Makespan is nine seconds versus 4+6+3+2=15 seconds for a fully serial order. The critical chain B→C is nine, and total work divided by two is 7.5, so the ideal bound is reached under these assumptions.

The result

The safe schedule exploits actual independence while respecting both prerequisites of C. It is a graph-specific result, not a blanket speedup from parallel calls.

Explore the model

Interactive lab

Interactive lab · queue

Concurrency still faces a service constraint

Reset, set λ=4 and μ=3, predict q(2)=6, then use four Steps. Separately Reset, set λ=4 and μ=5, predict q(2)=2, then use four Steps. This initial-four fluid queue has no dependency edges, worker slots, approvals, or provider API.

Concurrency still faces a service constraintFluid backlog starts at 4 and follows max(0, 4 + (λ − μ)t). The separate M/M/1 residence-time formula assumes Poisson arrivals, exponential service and λ < μ. Current readings: Fluid backlog: 4; Net rate · λ − μ: −1 /s; M/M/1 mean residence: 1 s. All parameters are adjustable with the labeled controls below.02.557.51001.534.56Elapsed time · sFluid backlogλ < μ
Fluid backlogCurrent time
Fluid backlog
4
Net rate · λ − μ
−1 /s
M/M/1 mean residence
1 s
t = 0 s

Fluid backlog starts at 4 and follows max(0, 4 + (λ − μ)t). The separate M/M/1 residence-time formula assumes Poisson arrivals, exponential service and λ < μ. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forBounded parallelism must preserve dependencies and match actual shared-resource capacity.

Check your understanding

Why can two read-only calls still need coordination before their results are combined?

Try it yourself

Independent A and B take two and five seconds. C takes four seconds and requires both. With two ideal slots and no overhead, find makespan and the fully serial duration.

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

C starts after the slower prerequisite finishes.

Compare with the solution

A and B finish by time five, then C finishes at nine. Fully serial work takes 2+5+4=11 seconds. The nine-second dependency chain B→C is the relevant ideal lower bound; real tool overhead or resource limits can add time.

Take it with you

  • Tool independence includes versions, effects, scopes, and shared resources.
  • Critical-path bounds need explicit scheduling assumptions.
  • Durable joins and structured ownership remain necessary under parallel completion.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.