Finalizers: make deletion wait for verifiable cleanup

For an object with finalizers, the API server sets deletionTimestamp on a deletion request and keeps the object until the relevant deletion conditions and finalizer removal are satisfied. The responsible controller observes the timestamp and performs cleanup instead of continuing ordinary provisioning. A successful delete request does not mean the object or external resource has already vanished. Once deletion is in progress, the API restricts changes to finalizers: existing ones can be removed, but new finalizers cannot be added to postpone an already requested deletion.

Register the controller’s finalizer before creating external resources that require it for cleanup, and check the successful persistence of that registration. Otherwise a crash or deletion between external creation and finalizer installation can orphan the external allocation. Even with the key installed first, external calls can have uncertain outcomes. Use stable identities tied to the object UID and queryable ensure/delete behavior so a retry can determine whether the intended cleanup already happened without targeting a new object incarnation. Custom finalizer keys must be qualified, such as greatitself.com/external-cleanup; short names in the example are symbolic labels.

Suppose deletion of an external resource succeeds but the controller times out before learning that fact. A retry should treat verified absence of that particular external identity as successful cleanup, rather than fail forever because a second delete finds nothing. Conversely, a timeout alone is not proof of absence. Keep the object and finalizer while the obligation is uncertain, report a useful condition or event, and retry with a bounded rate. Separate external identity lookup errors from a positive not-found result.

Remove only the finalizer key this controller owns after its obligation is satisfied. Other finalizers may represent independent storage or policy responsibilities. Update metadata with concurrency-aware logic so competing writers’ keys are preserved. Finalizer list order is not a reliable execution dependency mechanism; avoid designs where two controllers wait on each other’s key order. If an operator deliberately removes a stuck key, the procedure must account for the external resource and accepted data-loss or leak consequences, rather than treating force removal as a harmless retry.

Owner references let Kubernetes garbage collection manage dependent API objects under its ownership rules. They do not automatically delete an arbitrary cloud account resource or database entry outside the API. A finalizer can bridge that external lifecycle, but only if a functioning controller implements the bridge and has the required authority. Measure deletion age and finalizer failures separately from create latency. Test deletion during provisioning, controller restarts, denied external cleanup, and names reused with a new UID; these are ordinary lifecycle interleavings, not exotic accidents.

The queue lab illustrates a backlog of cleanup obligations under a stated service rate. Reset, set λ=2 and μ=3, predict backlog max(0,4−t), then Play or Step until t=4. The fluid queue counts abstract work only; it has no finalizers, deletion timestamps, API validation, or external confirmation. A backlog reaching zero cannot prove real resources were deleted. Use the example’s explicit confirmations for correctness and the lab only to reason about capacity when deletion requests accumulate.