Encrypting PII on a live system took nine steps

Part 1 of an ongoing series on Moty, a fitness data platform I build and run solo in production: one multi-tenant API, two Next.js frontends. All numbers come from production measurements and git history.

The data made the decision for me

I run the backend of a fitness platform built around a digital weight machine, the kind that streams force and velocity from both sides of your body, ten times a second. Most of that is exercise telemetry. But the platform also holds emails, phone numbers, and clinical-style rehabilitation metrics, and at some point "the database is encrypted at rest" stops being a satisfying answer. Disk encryption protects you from a stolen hard drive. It does nothing about a leaked dump, a misconfigured backup, or a query that ends up somewhere it shouldn't.

The platform is built under Korea's PIPA, and the engineering controls I wanted map one-to-one to (UK) GDPR vocabulary: security of processing, pseudonymisation, demonstrable consent. This post covers one of those controls, field-level encryption of PII in a production system, shipped in nine steps with zero downtime on a rolling deploy where old and new code briefly coexist on every release.

The problem is never the encryption

AES-256-GCM encrypts a field just fine. The problem is everything the plaintext column was doing for you on the side:

• Equality search. "Find the user with this email" is a login path. Ciphertext from GCM is non-deterministic (same input, different output every time), so WHERE email = ? is dead.
• Uniqueness. The UNIQUE constraint on email is a business rule. Non-deterministic ciphertext can't carry it.

The classic escape hatch is deterministic encryption, which gives you back equality at the cost of leaking equality itself: anyone with the ciphertexts can see which rows share a value. The design that shipped keeps GCM for the payload and adds a second column, an HMAC-SHA256 blind index. HMAC is deterministic, so equality and UNIQUE move to the index column; but without the key, the index reveals nothing and can't be reversed. So each field carries two columns. *_enc for the truth, *_bidx for the questions you're allowed to ask of it. Equality was the only query shape the product needed. I checked before committing to a design that can't do ranges.

Nine steps, because the deploy is rolling

A rolling deploy means every release has a window where the previous version and the new version run side by side against the same schema. Anything that isn't backward-compatible for that window breaks someone. So the rollout followed the expand→contract discipline, stretched across nine steps, each one a separate, traceable commit:

1. Crypto core. Encryption and blind-index primitives, tested in isolation, wired to nothing.
2. Feature gates, default off. Three switches: encryption on/off, backfill on/off, read-via-index on/off.
3. Additive schema. The new nullable columns. Old code doesn't know they exist; nothing breaks.
4. Dual writes. A persistence listener fills the encrypted columns alongside plaintext on every write.
5. Backfill batch. A scheduled job (locked so only one instance runs it) walks old rows in small batches.
6. UNIQUE on the blind index. Only possible once backfill completes; the constraint moves house.
7. Read path via the index. Lookups switch to the blind index behind the third gate.
8. Dev rehearsal. The full sequence executed end-to-end in the dev environment first.
9. Production, in three switch flips. Enable, backfill on, then backfill off and reads over.

The part I'd defend hardest in a design review is that the three switches live in the secrets store, not in code. Every phase transition, and every rollback, is a configuration change, not a redeploy. When you're one person operating production, "I can undo this without shipping" is the difference between doing the rollout on a Tuesday afternoon and doing it at 2am.

What nine steps buys you

Each step is individually boring. That's the point. At no moment does the system depend on something that doesn't exist yet: writes are dual before reads move, the constraint moves only after the data is complete, and the plaintext column keeps working until the very end. A failed step rolls back to the previous step, not to a backup.

The final contract step, actually dropping the plaintext columns, waited the longest, and that's deliberate. Contract is the only irreversible move in the whole sequence, so it shipped only after the encrypted path had soaked in production. One last migration removed the columns; what remains of those fields is the encrypted values and their blind indexes. Expand to contract, closed end to end.

The honest limits

• Key rotation is asymmetric. The payload keys sit under a managed KMS key with rotation; but rotating the blind-index key means re-indexing every row, because determinism is the feature. Rotating it is a planned batch job, not a config flip.
• The blind index leaks one bit by design, whether two rows share a value, and that happens to be exactly the bit UNIQUE needs. You're not choosing whether to leak it, only where.
• Encryption was the middle of the story, not the end. The same platform records consent in immutable logs (including refusals, because you can't prove you didn't get consent if you never wrote anything down), and, at account deletion, replaces analytics identifiers with unlinkable random ones, so cohort statistics survive while no key exists that could point back to a person. Field-level encryption is one control in a lattice, and the consent log was the harder design conversation. That's a story for another post.

Epilogue

Nothing in this rollout is novel. Expand→contract is folklore, and blind indexes are documented practice. What I'd pass on is the shape of the thing. The step count itself is what made this safe. Every time I was tempted to merge two steps, I was trading away a rollback point that cost nothing to keep. Nine boring commits beat one interesting one.