All posts

Why deleted data comes back: caches, search, and backup recovery

· 8 min read

Deleting the database row can leave copies and backups. Four tips connect TTL expiry checks, replay-safe deletion events, reapplying erasures after restoration, and throttling large deletion jobs.

Deleted data returns when copies are outside the deletion boundary

A departed user remains in search and reappears after recovery. The SQL inside a deletion API cannot settle the whole problem. This fictional membership service follows deletion from acceptance through caches, search, analytics, integrations, and backups. It translates an agreed deletion scope into implementation and checks rather than prescribing legal erasure deadlines.

Illustrative deletion flow: delivery alone is not completion

Record an authenticated, authorized request → Disable source use and durably record deletion → Apply deletion at each copy → Check destination results and remaining data → Retry and reconcile before recording completion

DestinationExample mechanismCompletion check
CacheInvalidate keys and prevent refill from stale readsTest reads and stale-update replay after deletion
Search and analyticsApply delete events or targeted ID deletionReconcile IDs and verify search visibility
External integrationUse its deletion and status APIsDistinguish acceptance from completion
BackupsDefine isolation, retention, and replay after restorationA restore drill must not expose erased records

Separate TTL cleanup from the time access stops

Expired DynamoDB items can remain readable before deletion. In addition to expiry filters on Query or Scan, check validity on direct reads, cached reads, and every use path. This case rejects data when now >= expires_at rather than waiting for physical deletion. Merely shortening TTL confuses the intended validity period with asynchronous cleanup.

Expiry pseudocode; define timestamp representation and acceptable clock error for the service
item = authorized_read(subject_id)
if item is missing or now >= item.expires_at:
    deny_use()
else:
    continue_with_authorized_operation(item)

Record access disablement, deletion from online copies, and the end of backup retention as separate states. Define a contract that can explain progress instead of hiding it behind one “deleted” label. Set retention from storage behavior and recovery needs; one TTL setting cannot establish erasure time for every copy.

Handle replay, stale updates, and failures in deletion propagation

Connect the source deletion to durable notification, using an outbox, CDC, or another suitable mechanism, so a crash cannot lose the deletion. Receivers use deletion identity and subject generation to replay safely and reject stale updates. Reusing an ID for a new member can let an old deletion erase new data; avoid reuse or distinguish generations.

The Debezium PostgreSQL connector distinguishes a row-delete event from the null-valued tombstone used for Kafka compaction. Follow the actual event format and configuration rather than inventing a blanket rule that every null means deletion. Record destination application results, not just delivery success, and alert on overdue work.

Restoration includes replaying deletion records

Restoring a January 1 backup on January 10 can revive data deleted in between. If 120 users departed in this example, reapply their subject identifiers and generations from separate records. Protect deletion records outside the database being rolled back. Retention must cover the oldest restorable point, detection of missing records, and possible replay windows.

  1. Isolate the restore target and stop automatic notifications, synchronization, and search exposure.
  2. Identify the restore point, collect subsequent deletion records, and check their completeness.
  3. Reapply deletions to the source and delete or rebuild affected derived copies.
  4. Verify erased identifiers cannot reappear before restoring access.

Google Cloud documents different deletion stages for active systems and backups. That does not guarantee automatic handling of every copy in your application. Deletion records themselves retain identifiers, so avoid retaining erased content and limit their purpose, access, and retention.

Batch large deletions and record durable progress

One huge DELETE can create lock contention, log pressure, and replica lag. Fix the target IDs, commit small ranges, and pause based on observed load and lag. Monitoring delay means a pause threshold is not a hard lag ceiling. Percona pt-archiver illustrates incremental work for MySQL; it is not a procedure to apply unchanged to other databases.

For an illustrative count, three million targets at 5,000 per batch require 600 batches. Measure batch time and pauses separately; counts alone do not promise completion time. Coordinate progress persistence and concurrent changes so older targets cannot slip behind the cursor.

Test restores, replay, and partial failures

  1. Stop one destination during account deletion; verify incomplete work is tracked and replayed after recovery.
  2. Replay an older update after deletion and verify the profile stays absent.
  3. At the exact expiry and before physical deletion, verify direct, list, and cached access reject use.
  4. Restore a pre-deletion backup in isolation; verify exposure is blocked before replay and erased data remains absent afterward, including search.

Primary sources

The official sources below were checked on October 27, 2026. Scenarios and numerical examples are illustrative assumptions. Configuration examples are explanatory; they have not been run against live services or benchmarked.

Tips this post digs into