Why distributed locks still allow stale writes: leases, clocks, and fencing tokens

October 27, 2026 · 9 min read

An expired owner can resume and write again. Four tips use a report job to explain destination-side generation checks, the difference from idempotency, and the clock and consistency assumptions behind safe handover.

Why can writes overlap after taking a distributed lock?

In a fictional report job, worker A obtains a ten-second lease and pauses for fifteen seconds. Its lease expires and B takes over. A may resume without recognizing the pause and overwrite B’s result. Distributed locking coordinates ownership; it does not physically stop a process when its lease expires.

Order	Worker and destination state	Unsafe result
1	A works under token 33, then pauses	A’s computation still exists
2	After lease expiry, B obtains token 34	B proceeds as the new owner
3	The destination recognizes 34 and publishes B’s result	The new result becomes visible
4	A requests publication late under 33	Without fencing, stale output can replace it

Checking the lock immediately before writing still fails if the process pauses between the check and the write. A longer lease changes frequency but does not eliminate arbitrarily long pauses or delays. Redis documentation recommends fencing, while client-go’s leader-election documentation explicitly disclaims that guarantee.

Is the lock protecting efficiency or correctness?

If the goal is to reduce duplicate expensive computation, two workers temporarily computing may be acceptable when only a valid result can be published. Protecting balances or a unique published result requires destination-side rejection of invalid writes. A lock reducing repeated work is not itself the business invariant.

Fencing works only when the destination checks the token

A fencing token is an ownership generation increasing across handovers. A trusted issuer allocates it, and the destination durably recognizes a newer generation before rejecting older ones. A wall-clock timestamp or random UUID does not establish handover order. Recovery of the issuer must not roll the numbering backward.

Destination-side pseudocode; serialize generation registration and writes on the same control row

with destination_transaction():
    lock_resource_control_row(resource_id)
    require supplied_epoch == current_epoch(resource_id)
    if not applied(operation_id):
        apply_business_change()
        record_operation(operation_id)

Execute the generation check, business change, and operation record as one atomic operation, so another operation cannot interleave between them. Reading the generation on the client before an unconditional write allows a new generation to register in between. Replays within one generation need an operation ID because the epoch alone cannot distinguish them. Return recorded results after response loss rather than executing a new business operation.

Do not determine lock ownership from a replica whose generation may be stale

If A and B both succeed after separate replicas identify their own generation as current, later convergence cannot undo an emitted report or payment. This destination requires generation registration and conditional updates to share one authoritative order. Linearizability makes an operation appear at one point between invocation and completion, unlike merely converging eventually.

A generation check inside one database does not automatically cover an external mail service or file write. If the external destination cannot enforce epochs, consider writing output to a temporary file and switching the published reference only if the generation still matches. The generation check and reference switch must also be atomic at the destination. For external APIs, also use business idempotency keys and result lookup, checking which failures each mechanism prevents and which conditions the destination can enforce. Adding a lock does not establish exactly-once effects across systems.

Leases and clocks: shorter does not mean safer

A shorter lease can speed handover after failure but also trigger handovers during normal pauses. A six-second pause exceeds a five-second lease even on an otherwise healthy worker. Actual handover includes detection, retries, communication, and election; lease plus retry is not a universal bound. Observe pause durations and handover frequency.

Measure durations with a monotonic clock. Java System.nanoTime supports elapsed-time differences within one JVM, not comparing timestamps from separate machines. A monotonic clock does not make a paused process progress, so destination fencing remains necessary. Wall-clock recency is not proof of current ownership.

Test the former owner resuming after a pause

Pause A after acquisition but before writing; let B register a new generation after expiry.
Publish B’s result, resume A, and verify destination-side rejection of the old epoch.
Pause between generation checking and mutation, and between application and response; verify atomicity and replay by operation ID.
Test issuer and destination recovery, checking epochs do not roll back and external effect outcomes remain discoverable.

Primary sources

The official sources below were checked on October 27, 2026. Scenarios and numerical examples are illustrative assumptions. Configuration examples are explanatory; they have not been run against live services or benchmarked.