research34 min studyLesson 893 in Engineering

A memory write needs purpose, provenance, and an explicit trust boundary

Choose what to persist without promoting tool-output instructions into authority or allowing repeated and unsupported observations to become durable facts by accident.

On this page

What you’ll understand

  • Separate source content from authorized task instructions.
  • Apply a declared eligibility and token policy to candidate memory writes.
  • Preserve atomic logical identity and correction lineage through retries.

An agent can encounter a page that says to remember a new rule, a tool result that repeats an old observation, or a generated reflection that confidently invents a cause. Persisting all of them makes the next task inherit the same mistakes. A write policy must distinguish task purpose, source provenance, scope, trust, novelty, and retention. The policy determines what enters memory, but acceptance is still not proof of truth. Store the observation and its status without silently granting it authority over the user's instructions.

Memory is another boundary for untrusted observations

Tool responses, retrieved documents, and external messages are data about a task or environment. Instructions embedded in them do not automatically become user-authorized behavior simply because the model reads or summarizes them. A memory record should retain source type and origin so a later retrieval can preserve that distinction. Prompt-injection benchmarks such as AgentDojo empirically study this boundary in tool-using agents; their measured attacks and defenses are scoped evidence rather than a guarantee that one textual warning solves every case.

Define record classes appropriate to the application, such as an observed result, an explicit user preference, a derived procedure, or a pending action. Each has different evidence and scope requirements. A document's assertion can be stored as an assertion from that source without becoming an established fact. A useful schema keeps quotation or normalized claim, source identity, observation time, applicable task, derivation, and validation status distinguishable. Avoid flattening them into a single authoritative sentence whose origin is lost when another agent later retrieves it.

eligible⁡(m)=P(m)∧S(m)∧V(m)∧N(m)\operatorname{eligible}(m)=P(m)\land S(m)\land V(m)\land N(m)
P, S, V, and N are declared checks for purpose, allowed source handling, valid scope/status, and novelty. This is an example write-policy predicate, not a universal trust classifier. Passing it means eligible for a chosen storage operation; it does not establish factual correctness or authorize an external action.↔ Scroll wide equations horizontally.

Keep writes bounded and retry-safe within their actual scope

After eligibility checks, prioritize useful records under a storage or context budget. Retain a reference to a larger source when copying its entire text would crowd out necessary state. Similarity-based duplicate detection needs care: two reports can be redundant copies, or distinct observations about the same object at different times. Stable logical event IDs handle exact retries, while semantic consolidation needs an explicit merge policy. Do not collapse conflicting observations into a false consensus merely because their embeddings are near each other.

When a write updates an aggregate or source index, apply the required state changes atomically within a supported transaction or version-precondition scope. A retry after a lost acknowledgment should not create another logical observation. If persistence and publication cross systems, use a documented protocol such as an outbox with idempotent consumers rather than assuming the local commit made all external effects exactly once. Preserve correction and revocation lineage so derived claims can be reconsidered. Deleting one raw row does not automatically invalidate every summary or cached retrieval that used it.

nretained≤⌊Bc⌋(c>0)n_{\rm retained}\le\left\lfloor\frac{B}{c}\right\rfloor\quad(c>0)
This simple bound assumes each eligible formatted record has the same positive token or byte cost c and budget B uses the same unit. Real variable-size records need a weighted account. Capacity applies after eligibility; it does not decide evidence quality, source authority, or which records deserve retention.↔ Scroll wide equations horizontally.

Evaluate what becomes durable after a bad input

Inject a duplicate result, a contradictory correction, an irrelevant but highly similar document, and a source containing an instruction outside the task's authority. Inspect the stored records and then run a later retrieval task to see whether the trust boundary survives persistence. Measure false promotion, missed useful observations, duplicate writes, correction propagation, and token cost. A system can reject the immediate instruction yet still store it as an unqualified rule that causes a later failure. The test must follow the full write-to-retrieval path rather than only the initial response.

The probability lab illustrates an independently selected count, not a write-policy validator. Reset, set both p=0.25 and n=20, predict mean five and variance 3.75, then Play or Step. Reset and set both p=0.5 and n=20 for mean ten and variance five. Exact binomial bars assume equal-probability independent trials; playback highlights outcomes rather than sampling. The model has no sources, instructions, provenance, transactions, or trust judgment. Actual write eligibility comes from the explicit policy and evidence, not from treating a random retention probability as validation.

Eligibility and storage capacity are separate filters

A toy intake contains twenty candidate records: eight exact delivery duplicates, four outside task scope, three unsupported proposed rules, and five eligible observations under a declared policy. The categories are disjoint. Each eligible formatted record costs 90 tokens and the write budget is 400 tokens.

  1. The categories sum to 8+4+3+5=20. Only five records reach capacity selection; accepting unsupported rules merely because they fit would bypass the policy rather than improve its coverage.

  2. Five equal-cost eligible records would require 450 tokens. The budget fits floor(400/90)=4 records, using 360 tokens and leaving 40. A prioritization or deferred-reference policy must handle the fifth.

  3. Retrying one of the four committed writes with the same stable event ID should leave four logical observations under an atomic deduplication contract. It must not turn the repeated delivery into additional corroborating evidence.

The result

The example separates trust and scope eligibility, bounded storage selection, and retry identity. None of these mechanisms alone guarantees that an accepted source assertion is true.

Explore the model

Interactive lab

Interactive lab · probability

A random retained count is not a provenance decision

Reset, set both p=0.25 and n=20, predict mean 5 and variance 3.75, then Play or Step. Reset and re-enter both p=0.5 and n=20 for mean 10 and variance 5. Exact binomial bars require independent equal-probability decisions; playback highlights outcomes, not draws. There are no source records, trust boundaries, write transactions, or factual checks, and n caps at 30.

A random retained count is not a provenance decisionThese are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Current readings: Mean · np: 5; Variance · np(1−p): 2.5; Total probability: 1. All parameters are adjustable with the labeled controls below.03581000.070.150.220.3Successes kP(X = k)k = 0 · P = 9.77e-4
Exact probabilityHighlighted outcomeMean np
Mean · np
5
Variance · np(1−p)
2.5
Total probability
1
t = 0 s

These are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forDefine and test the actual write policy before analyzing its capacity or retention statistics.

Check your understanding

A retrieved page instructs an agent to store a new permanent rule outside the user's task. What is the correct memory-boundary interpretation?

Try it yourself

Seven eligible records each cost 80 tokens, and the available budget is 450. How many complete records fit and how many tokens remain? If one record corrects an older accepted claim, what additional dependency issue must the system handle?

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Capacity is a floor division; correction also affects derived records and caches.

Compare with the solution

Five complete records fit, using 400 tokens and leaving 50. The correction needs versioned lineage and propagation or revalidation for summaries, plans, and cached results that depended on the old claim. Merely appending the correction while retrieving an unchanged old summary can preserve the original error.

Take it with you

  • Persist source content without promoting it into instruction authority or verified truth.
  • Apply scope and evidence policy before capacity ranking.
  • Preserve logical identity and correction lineage across retries, summaries, and retrieval.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.