How to search encrypted data without decrypting it

Client-side encryption gives you a real guarantee, the provider can't read your data, but it breaks the thing every app does constantly: look a record up by one of its fields. If email is stored as ciphertext, WHERE email = 'sarah@example.com' can't work, because the stored value doesn't equal the search value. The technique that fixes this is the blind index.

Why you can't just search the ciphertext

Good encryption is randomised: encrypting "sarah@example.com" twice produces two completely different ciphertexts (a fresh random nonce each time). That's exactly what you want, it stops anyone learning that two rows hold the same value. But it also means ciphertext is useless for matching: two encryptions of the same email don't look equal, so the database can't group or find by them. You need something that's the same every time for the same input, but still reveals nothing.

Enter the blind index: a keyed one-way hash

A blind index is a keyed hash of the plaintext, typically HMAC-SHA256, computed with a secret key only you hold. Two properties make it work:

• Deterministic: the same input always produces the same hash, so equal values match.
• One-way and keyed: you can't reverse the hash to recover the value, and without the key you can't even compute it to test a guess cheaply at scale.

So each sensitive row stores two things: email_enc (randomised ciphertext, for reading back) and email_bidx (the deterministic hash, for finding). Your app computes the hash before it queries; the database only ever compares hashes.

What a search actually looks like

To find a customer by email, the flow is: your application normalises the term (lowercase, trim), computes HMAC(key, "sarah@example.com"), and sends WHERE email_bidx = '<hash>'. The database returns the matching row as ciphertext; your app decrypts it locally with the encryption key to display it. The provider ran the WHERE, matched on a hash, and never saw either the search term or the result in the clear. You can watch this happen step by step, with timings, in the interactive demo.

What it leaks, be honest about this

Blind indexes are a deliberate, bounded trade of a little privacy for the ability to search. The leakage is real and worth understanding:

• Equality is visible. Because the hash is deterministic, the database can see that two rows share the same value (same hash), even though it can't tell what that value is. For a unique field like email that's minor; for a low-cardinality field it matters.
• Low-cardinality fields leak through frequency. Blind-indexing a field with few possible values, say a status or a postcode, lets someone count how often each hash appears and often infer the values from the distribution. The rule: don't blind-index low-entropy fields. Prefer a high-entropy unique identifier (an email, a member number) as your search key.
• Guard the index key. The blind-index key must be protected like the encryption key. If it leaks, an attacker who can guess candidate values can confirm which rows hold them.

What it can't do (yet)

A basic blind index is exact-match only. It finds "sarah@example.com", not "sarah%", not a range, not a substring, because a hash of the whole value bears no relation to a hash of a part of it. You can extend it, hashing normalised components (first name, last name) as separate indexes to narrow searches, or use more advanced order-revealing schemes for ranges, but each extra capability reveals a bit more structure. Exact match, done well, covers the overwhelming majority of real lookups: login by email, find by ID, dedupe by phone.

Where this fits on WattleDB

Searchable client-side encryption is coming to WattleDB as an opt-in feature, built on the open-source, audited Acra project, which implements exactly this AES-GCM-plus-blind-index model. You keep the keys; WattleDB stores ciphertext and hashes on Australian-owned infrastructure. Register for early access or try the demo.