foundation17 min studyLesson 9 in Engineering

From source code to a controlled release

Separate builds, artifacts, configuration, migrations, and rollout decisions so a change can be verified and recovered.

On this page

What you’ll understand

  • Distinguish building an artifact from deploying and releasing it.
  • Explain artifact identity and reproducible-build scope.
  • Plan a rollout with evidence, compatibility, and a recovery path.

A successful compilation says that a build step completed. It does not say that users can reach the new version, that its data migration is safe, or that its behavior improved. Release engineering joins these stages into a reviewable process. The goal is to make the running change identifiable, its impact observable, and the response to trouble deliberate.

Build once, identify what runs

A build transforms source and declared inputs into an artifact such as a package, executable, or container image. A deployment places a selected artifact into an environment. A release exposes behavior to users, sometimes through a separate traffic or feature decision. Keeping these concepts distinct helps explain how a staged deployment can exist without serving all users and why rebuilding separately for each environment can introduce unintended differences.

An artifact digest identifies bytes; it does not prove those bytes are correct or trustworthy. Record source revision, dependency resolution, build inputs, and relevant provenance. A reproducible build means that specified inputs and environment can produce the same artifacts according to the project's definition. It is narrower than a claim that arbitrary machines, unspecified timestamps, and changing dependencies will always produce identical output.

Nexposed=fNN_{\rm exposed}=fN
A rollout fraction f exposes approximately f times N eligible requests or users over a stated window. The counted unit must be consistent; sticky user assignment and per-request routing produce different exposure patterns.↔ Scroll wide equations horizontally.

Configuration and data can change the result

The same artifact can behave differently with different configuration, permissions, network dependencies, or data. Validate configuration without printing secrets, and document which values are safe to change at runtime. An environment variable is a delivery mechanism, not a substitute for a clear configuration contract. A health endpoint can show that a process responds while a critical user workflow still fails, so readiness and user-level checks need separate attention.

Data migrations deserve their own compatibility plan. During a gradual rollout, old and new versions may run simultaneously. A change that immediately removes a field required by the old version can break a rollback. An expand-and-contract approach can introduce compatible structure first, migrate and verify data, then remove obsolete structure later. It remains a pattern to adapt and test, not a guarantee that every schema change becomes reversible.

Eadditional=Nf(pnew−pold)E_{\rm additional}=Nf(p_{\rm new}-p_{\rm old})
For comparable traffic and empirical error fractions, this estimates additional errors over N eligible requests at rollout fraction f. It is a planning approximation, not causal proof; traffic differences and uncertainty can dominate small samples.↔ Scroll wide equations horizontally.

A rollout is an experiment with an exit condition

Before rollout, define success signals, stop conditions, an observation window, and who or what can act on them. Compare errors, latency, and relevant product outcomes against a suitable baseline. A tiny rollout may reduce exposure while supplying too little evidence about rare failures. A quiet dashboard can mean success, insufficient traffic, or broken telemetry. Check that the measurement itself is functioning.

Recovery might mean routing traffic back, redeploying a known artifact, disabling a feature, or restoring compatible data. Choose it before an incident rather than assuming every release can be undone by changing a code pointer. In the lab, vary total replicas and unavailable replicas to inspect a majority threshold. This illustrates only how remaining instance counts compare with a threshold; a release may need a different readiness policy. It does not deploy software or show data compatibility, request capacity, correlated failure, or statistical experiment design. State those missing checks before interpreting an online count as release evidence.

Interpret a five-percent canary

A service expects 100,000 eligible requests in an observation window. A five-percent canary has an observed error fraction of 0.02, compared with a baseline of 0.005 on comparable traffic.

  1. Estimate canary exposure: 0.05 × 100,000 = 5,000 requests. At the observed fraction, that corresponds to about 100 canary errors.

  2. At the baseline fraction, the same request count would yield about 25 errors. The simple excess estimate is therefore 75 errors, assuming comparable traffic and adequate measurement.

  3. Apply the predeclared stop policy and investigate. Verify traffic comparability and telemetry; inspect a tested recovery path and any data compatibility constraints rather than treating the arithmetic as definitive causal attribution.

The result

The canary limits exposure and supplies a warning signal. It still requires interpretation and a prepared recovery action.

Explore the model

Interactive lab

Interactive lab · reliability

Online replicas are only one rollout signal

Set replica count n and unavailable count k, then compare remaining replicas with a majority. The count does not show their versions, capacity, correctness, or data compatibility, and majority availability alone does not establish safe deployment.

Online replicas are only one rollout signalA replica count and a majority threshold, without a consensus protocol. Playback highlights fixed replica states. Having a majority alone does not establish linearizability, durability or failure independence. Current readings: Available replicas: 4 of 5; Majority needed: 3; Majority available?: Yes. All parameters are adjustable with the labeled controls below.Majority available · need 3 of 5Playback highlights fixed statesR1 · upR2 · upR3 · upR4 · upR5 · downCyan mark: majority threshold
AvailableUnavailableMajority threshold
Available replicas
4 of 5
Majority needed
3
Majority available?
Yes
t = 0 s

A replica count and a majority threshold, without a consensus protocol. Playback highlights fixed replica states. Having a majority alone does not establish linearizability, durability or failure independence. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forInstance counts help describe exposure to outages, but a release needs behavioral evidence and a recovery plan.

Check your understanding

Why might switching back to an old artifact fail to recover a service?

Try it yourself

A one-percent rollout receives 300,000 eligible requests. Estimate exposure and explain why zero observed errors is not proof of zero risk.

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Multiply the request total by 0.01, then distinguish an observation from a universal claim.

Compare with the solution

Approximately 3,000 requests reach the canary under per-request allocation. Zero observed errors means none were detected in that sample and window. Rare paths, different users, later failures, or missing telemetry can remain untested; increase exposure only under a reasoned observation and recovery policy.

Take it with you

  • Build, deployment, and release are related but distinct stages.
  • Identify artifacts and record the inputs that produced them.
  • Plan rollout evidence and data-compatible recovery before exposure.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.