advanced25 min studyLesson 244 in Engineering

Replicated logs: match a prefix before extending it

Trace AppendEntries consistency checks and conflict repair while distinguishing stored suffixes from committed state-machine history.

On this page

What you’ll understand

  • Explain the index-and-term predecessor check in AppendEntries.
  • Repair a conflicting uncommitted suffix without truncating a matching prefix.
  • Distinguish replication backlog from the proof that two logs agree.

A follower may miss messages, retain entries from an old leader, or restart with a shorter durable suffix. A new leader cannot safely append commands by asking whether the follower's file merely has enough bytes. Raft matches logical log positions and terms, then repairs conflicts according to its protocol. The central result is a shared prefix, not permanent equality of every replica's entire log. We work with correctly formed entries and the full election and persistence assumptions established earlier.

A predecessor identifies the expected prefix

AppendEntries includes the index and term of the entry immediately before the entries being sent. A follower first checks the request term and the predecessor match. If the predecessor is absent or has a different term, the follower rejects rather than pretending an extension can be spliced onto an unrelated history. The leader then adjusts its replication position for that follower; implementations may return conflict hints to avoid retrying one index at a time. Such optimizations must preserve the same matching requirement.

Raft's log-matching property says that if two logs contain an entry with the same index and term, their entries agree through that position. The argument relies on one legitimate leader per term creating entries consistently and on followers accepting extensions only after a matching predecessor. Comparing two arbitrary files with the same numeric annotation would not prove their bytes equal. Likewise, a checksum detects some corruption but does not replace the protocol's history invariant or its non-Byzantine assumptions.

term⁡A(i)=term⁡B(i)⟹LA[1:i]=LB[1:i]\operatorname{term}_A(i)=\operatorname{term}_B(i)\quad\Longrightarrow\quad L_A[1{:}i]=L_B[1{:}i]
This is Raft's log-matching property for entries at the same existing index i under the complete protocol invariants. L[1:i] denotes the inclusive prefix of abstract entries, with dimensionless indices. Equal metadata on arbitrary corrupted logs does not establish the property.↔ Scroll wide equations horizontally.

Conflict repair is conditional, not blind truncation

After a successful predecessor check, a follower compares incoming entries with any entries at those indices. At the first same-index, different-term conflict, it deletes that conflicting entry and its following suffix, then appends the incoming continuation. Entries that already match need not be rewritten. A heartbeat or a shorter duplicate request must not automatically delete an otherwise valid longer suffix merely because the request did not mention it. The trigger is an actual conflict with a supplied entry, not the end of a message.

The leader maintains follower-specific progress estimates and updates them from responses in the relevant term and request context. Delayed failure replies must not undo later knowledge carelessly. A follower's successful response represents the storage state required by the implementation's durability contract. An entry stored on one follower is not necessarily committed, and a repaired follower must apply only the committed ordered prefix. Correct Raft never repairs disagreement by overwriting an already committed command with a different command at the same index.

j=min⁡{i:term⁡local(i)≠term⁡incoming(i)},Lrepaired=L[1:j−1]∥E[j:]j=\min\{i:\operatorname{term}_{\rm local}(i)\ne\operatorname{term}_{\rm incoming}(i)\},\qquad L_{\rm repaired}=L[1{:}j-1]\mathbin{\Vert}E[j{:}]
After a matching predecessor, j is the first actual conflict among supplied overlapping entries. The concatenation keeps the matching prefix and replaces the conflicting suffix. If no conflict exists, this truncation rule does not fire; missing incoming entries can simply be appended.↔ Scroll wide equations horizontally.

Replication rate cannot establish a matching history

Follower catch-up has both correctness and capacity concerns. A leader generating entries faster than a follower can durably receive them creates lag, even if every AppendEntries exchange is correct. Batching, flow control, and snapshots can help, but must not let a follower acknowledge data it has not safely processed. Diagnose the longest blocked stage: transport, persistence, or application. Each lag metric describes a different boundary, and none can certify that a conflicting suffix was repaired correctly.

Use the queue lab only for a normalized replication-lag analogy. For a growing fixed-rate run choose Reset, explicitly set arrival four and service three, predict the backlog slope, then Play or Step. For a separate draining run Reset and explicitly set arrival two and service three before predicting and playing. Rate edits preserve elapsed time and recompute from four initial items; they do not continue a prior backlog with a new rate. The lab contains no log entries, terms, predecessor checks, or conflict repair.

Replace the first real conflict

A leader and follower share entries through index three with terms [1,1,2]. The leader's full terms are [1,1,2,4,4,4], while the follower's are [1,1,2,3,3]. An accepted AppendEntries request names predecessor (3,2) and carries leader entries four through six. All conflicting entries are uncommitted.

  1. The follower has predecessor index three with term two, so the predecessor check succeeds. No earlier entry is removed: indices one through three are the common prefix.

  2. Incoming index four has term four while the local entry has term three. This is the first supplied conflict, so the follower deletes its indices four and five.

  3. Append incoming indices four, five, and six, producing [1,1,2,4,4,4]. The follower now has six entries. A later duplicate request containing only a matching index four would not justify deleting matching indices five and six.

The result

Conflict repair preserves the three-entry prefix and installs the leader's continuation. This storage result does not alone say which entries are committed or applied.

Explore the model

Interactive lab

Interactive lab · queue

Catch-up capacity without a log simulator

Compare separate fixed-rate runs: Reset, explicitly set both λ=4 and μ=3, predict, then Play or Step; Reset again and explicitly set λ=2 and μ=3 for draining. Rate edits retain elapsed time and recompute from four initial items. The fluid model has no Raft indices, terms, matching check, or conflict repair.

Catch-up capacity without a log simulatorFluid backlog starts at 4 and follows max(0, 4 + (λ − μ)t). The separate M/M/1 residence-time formula assumes Poisson arrivals, exponential service and λ < μ. Current readings: Fluid backlog: 4; Net rate · λ − μ: −1 /s; M/M/1 mean residence: 1 s. All parameters are adjustable with the labeled controls below.02.557.51001.534.56Elapsed time · sFluid backlogλ < μ
Fluid backlogCurrent time
Fluid backlog
4
Net rate · λ − μ
−1 /s
M/M/1 mean residence
1 s
t = 0 s

Fluid backlog starts at 4 and follows max(0, 4 + (λ − μ)t). The separate M/M/1 residence-time formula assumes Poisson arrivals, exponential service and λ < μ. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forA follower must both keep up with offered work and obey the log-matching protocol; a backlog curve proves neither prefix agreement nor commitment.

Check your understanding

A follower accepts a request whose predecessor matches, and all supplied overlapping entries also match. The request is shorter than the follower's existing matching suffix. What truncation is justified solely by that request?

Try it yourself

A follower has terms [1,2,2,5] at indices one through four. A request names predecessor (3,4) and supplies an entry at index four. Should it splice that entry in? How does the answer change if the predecessor is (3,2) and the supplied index-four entry has term six?

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Check the predecessor before comparing the new entry. A term mismatch at the predecessor is a rejection condition, not a request to guess a prefix.

Compare with the solution

The first request fails because the follower's index-three term is two, not four; it must reject without splicing. With predecessor (3,2), the check succeeds. If the supplied index-four term six is a valid continuation from the legitimate leader and the local conflicting entry is uncommitted, the follower replaces index four's term-five entry with the incoming term-six entry. A successful storage update still needs the appropriate persistence before acknowledgement and a separate commitment decision before application.

Take it with you

  • Match an index-and-term predecessor before extending a follower log.
  • Only an actual supplied conflict triggers suffix replacement.
  • Stored, committed, and applied positions describe different progress boundaries.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.