Actor-learner transport: throughput needs trajectory identity

Give a fragment an actor identity, actor incarnation, environment instance, episode identity, timestep interval, behavior version, and schema version. An actor restart can reuse a local counter, so actor name plus local sequence number alone may collide with an old in-flight message. Retried delivery should refer to the same immutable fragment, not freshly sampled data under the same key. Validate payload length, masks, action probabilities, and recurrent boundaries before admission. A replay buffer or learner must decide explicitly whether duplicates are ignored, charged, or intentionally reused.

Out-of-order arrival is normal in a parallel system. Reassemble sequences only when the identifiers prove adjacency within the same episode and compatible behavior/history contract. A gap can end a fragment’s local recursion and require a suitable bootstrap; it cannot be filled by attaching the next packet from the same numeric worker slot. Publish policy snapshots atomically so actors do not observe mixed parameters. Define acknowledgment durability and retention if exactly-once admission is claimed, because transport retries and a process-local seen-set do not by themselves establish that property across crashes.

When production exceeds learner admission, increasing actor count can increase queue age rather than useful policy improvement. A bounded queue forces a choice: block actors, reject fragments, drop older data, or use an algorithm whose replay and correction assumptions support retention. Each option changes resource use or the experience distribution. Dropping by age can favor faster environments; blocking can alter exploration throughput. Track valid transitions, admitted fragments, rejected data, behavior-age distribution, and task coverage rather than declare success from actor frames per second alone.

Size transport in bytes as well as transitions. Images, action masks, log probabilities, recurrent state, and duplicated observations can dominate message size. Compression or reduced precision needs a semantic and numerical contract; it cannot silently change observations or rewards used by the learner. A large queue of old-policy trajectories also consumes bandwidth for data that a strict on-policy batch may reject. Bound the end-to-end path, including receiver staging, replay storage, and learner prefetch, so moving data out of one visible queue does not hide another unbounded buffer.

Use a deterministic small environment with tagged episodes and scripted actor restarts to test missing, duplicated, delayed, and reordered fragments. Confirm that priority updates target the sampled item generation and that batch filters preserve the intended behavior versions. Measure how a learner slowdown propagates to actor queues and data age, then verify recovery without joining unrelated trajectories. Report dropped data and its task distribution. A transport can have excellent delivery reliability while violating the policy-learning protocol through stale or misattributed experience.

The queue lab supplies a limited overload analogy. Reset, explicitly set arrival four and service three, predict growth from four items, then Play or Step. For a separate run Reset and explicitly set arrival two and service three before predicting drainage. Rates are fixed for each run; edits retain time and recompute from the initial four. The fluid instrument has no actors, fragment identity, finite-capacity drop policy, byte sizes, or policy versions. Use the actual transport fixture to test the mechanisms it omits.