My notes say “Mike.” My calendar has an attendee id that might be him. A few weeks ago I wrote a short post about why there is no right way to join those two, and why the honest answer has three outcomes instead of two: link, no-link, and review. (That post is here.)
That post was about where the idea came from. This one is about the third outcome. The first two are cheap. The third one has a reader, and the reader is me.
What follows is a design, not a running system, and I’ll describe it as one.
Where this sits
I keep a store of facts my systems read from: who people are, which machine does what, what I decided and when. The trouble with a store like that isn’t that it’s wrong on day one. It’s that it drifts from the world without saying so. I ended up with a list of seven separate ways that drift happens. This is the fourth: object linking, joining the nouns in my notes to the keys each system actually uses. “Mike” in a note; an email address in Gmail; an attendee id in Calendar; a user id in Slack; a hostname on the network.
The seventh item on the list is attention. I didn’t expect the fourth to lead to the seventh. It does, and that link is the whole argument of this piece.
The design, briefly
Three tables.
An entity is the thing itself: a person, an organization, a device, a project. It has a label and a status: active, merged, or retired.
A designator is one way some system refers to that entity: a system, a key type, a key. It carries a confidence, the dates it was first and last seen, and the evidence for it. The same email address can only point at one entity. Designators that go unseen for 180 days are marked stale and stop counting until they’re seen again, because otherwise the registry becomes its own source of stale facts.
A discriminator is an attribute worth comparing, such as email, phone, display name or household, with two numbers attached: how often it agrees when two records really are the same thing, and how often it agrees by coincidence when they aren’t.
The resolver takes a new observation, adds up the evidence from every attribute that agrees, and returns one of three verdicts. Above an upper threshold, link. Below a lower threshold, no-link. In between, review.
None of this is new. It’s probabilistic record linkage, published in 1969, and the parts around it are standard data-management practice with the serial numbers filed off. The only unusual thing is the setting: one person, no schema owner, and every error lands on the person doing the reviewing.
The two errors aren’t the same size
There are two ways to get a join wrong.
A false split is when Mike shows up as two people. Annoying. I see two Mikes, I notice, I fix it. The error announces itself.
A false merge is when two different people become one Mike. Every fact about either of them now attaches to both. Nothing fails, nothing errors, and each downstream answer is individually plausible. It’s wrong and quiet in exactly the way that’s hardest to catch.
So the upper threshold is set conservatively, on purpose. When the system isn’t sure two records are the same person, it doesn’t guess yes. It asks.
And merges are never deletions. A merge marks one entity as merged and points it at the other. The rows stay. If I merged wrongly, I can split again, because the evidence I merged on is still there.
The review bucket is not a defect
Here’s the part I got wrong the first time I thought about this. I treated “review” as the failure case, the pile the algorithm couldn’t handle, something to shrink.
It isn’t. The review bucket is the only honest place in the whole design. It’s where the system says I don’t know out loud, instead of rounding to a guess. Every threshold choice is really a choice about how much of that pile I’m willing to read.
Widen the band and I get fewer silent errors but more reviews, and my attention is finite and already spoken for. Narrow it and the reviews drop, but the maybes don’t go away. They get auto-decided, some of them wrongly, and nobody is told.
A system without a review bucket isn’t cheaper. It bills later, and quietly.
So the design measures its own cost from day one: every review logs how long it took me to decide. If reviews per week are still trending up after the first month, the thresholds are wrong. That’s the failure state, written down before the thing is built, so I’ll know it when I see it rather than getting used to it.
What a review is worth, if you write it down
A review decision has a second value, and it disappears if you don’t catch it at the time.
Each time I look at a maybe and say “yes, same Mike” or “no, different Mike,” I’m producing a labelled example: this combination of agreeing and disagreeing attributes turned out to be the same person. Collect enough of those and the hand-set numbers in the discriminator table can be replaced with measured ones. No separate data-collection project needed.
But only if each decision records what was known at the moment:
which attributes agreed, which disagreed, and which were simply absent. Absent isn’t disagreement, and collapsing the two poisons the data.
the score, and both thresholds in force.
every candidate considered, not just the one I picked.
my verdict, and which version of the weights produced the score.
That last one matters more than it looks. The moment I change a weight, every earlier decision was made under different arithmetic. A log that doesn’t say which weights it used can’t be replayed.
And none of this can be rebuilt later. Six months on, the designators behind today’s decision may have gone stale and dropped out. The agreement vector I’d need is gone. Skipping the logging doesn’t postpone the learning. It gives it up.
Where I’d look first
One more thing, and I learned it the hard way.
On an earlier pass over the same store, I sent the classes that seemed consequential to human review and auto-accepted the rest. The costly errors were sitting in the part I auto-accepted. I’ve written that one up separately. There was a second lesson in that pass. I reviewed 44 flagged items and overrode none of them. A review that never disagrees isn’t evidence the machine is right. It might be evidence that the reviewer is tired.
Apply that here and it points somewhere uncomfortable. In this design the expensive error, the false merge, doesn’t live in the review bucket. It lives in the link bucket, the part that’s auto-accepted because the score was high. The conservative threshold makes that bucket smaller. It doesn’t make it empty.
The design as written spot-checks nothing above the upper threshold. It should. A handful of confident links, sampled and looked at each month, is the cheapest insurance against the one error that never announces itself.
The cost is the point
The pitch for most automation is that it takes a human out of the loop. This design does the opposite on purpose. It decides where the human goes, puts a number on what that costs, and writes down what would show the number is wrong.
Someone has to look at the maybes. The only choice is whether you’ve budgeted for it or whether you find out what it cost later.