Deterministic vs probabilistic record matching
Deterministic matching uses exact rules; probabilistic matching scores similarity. When each fits, and how the two combine in modern ER systems.
Deterministic and probabilistic are the two ways a record-matching system decides whether two rows are the same entity. Most production record linkage systems use both.
Deterministic matching
A deterministic matcher applies a fixed rule. If email is identical, the records match. If tax_id is identical, the records match. If the SHA-256 of (normalized_phone || dob) collides, the records match. Each rule is a yes/no test that either fires or does not.
Deterministic rules are fast, fully explainable, and trivial to audit. A reviewer can see "these two rows merged because rule R3 fired on email" and decide whether R3 was reasonable. The downside is brittleness: jsmith@acme.com and j.smith@acme.com are different strings, so a pure email-equality rule misses them entirely.
Probabilistic matching
A probabilistic matcher scores how similar two records look across multiple fields and combines those per-field scores into a pair-level number between 0 and 1. Fuzzy string comparisons (Jaro-Winkler on names, token-set on addresses, phonetic encoders on first names), weighted by how much each field tells you, drive the score. A high score triggers merge; a middle score lands in review; a low score keeps the records separate. The math under most production probabilistic matchers traces back to the Fellegi-Sunter model from 1969.
Probabilistic matching handles the cases deterministic rules cannot reach. It also makes mistakes deterministic rules never would, which is why thresholds and review queues exist.
When to use each
Deterministic fits when your data carries a stable, unique identifier like a tax ID, a national patient ID, or a verified email. Probabilistic fits when you have only soft signals (names, addresses, phone numbers) and the input is dirty. Real systems run deterministic rules first as a high-confidence pre-pass, then hand the remainder to a probabilistic scorer. Golden Suite layers both: exact rules fire automatically on identifier fields, and the remaining ambiguous pairs run through the configured similarity metrics and threshold.