Data quality · Entity resolution · Data operations
How do you keep entity matching dependable as data changes?
Define the identity being matched, evaluate incorrect and missed links, and give classification changes and corrections an explicit owner.
Decide what a match actually means
Begin by naming the object your product needs to recognize. A publication, an edition and an author are different entities. In a business system, an organization, a branch and an account can also have different identifiers. Establish the required level before merging records with similar descriptions.
Crossref’s DOI guidance separates a persistent identifier from the metadata describing its object and warns that a DOI does not establish the object’s accuracy or value. Use an identifier within its documented scope. Keep descriptive fields and their sources available for verification and correction.
When identifiers are missing or conflict, retain the original records and make the proposed match inspectable. Define the additional evidence needed to accept it. Give ambiguous cases an unresolved state so downstream users can distinguish a confirmed link from a convenient assumption.
Measure incorrect links and missed links separately
The UK government’s data-linkage quality guidance distinguishes false links from missed links and explains that the appropriate tradeoff depends on the use. Review both. Combining two unrelated records can contaminate a history; failing to join two versions of the same record can duplicate counts.
Build checks around the records your process actually receives, including new sources, missing fields and plausible lookalikes. Inspect quality by source and record type as well as overall. A favorable average may leave the subset used for an important decision poorly matched.
Human review needs enough evidence to resolve the case. The same guidance notes that experienced reviewers can make mistakes and that reviewing likely matches alone is weak evidence about missed matches. Record reviewer disagreements and design a separate check for records the system left unlinked.
Maintain classifications without erasing their history
Treat identity and classification as separate decisions. A record can keep its identity while its category changes. W3C’s SKOS standard distinguishes concepts, labels and relationships, and includes properties for definitions and change history. That separation is useful even when a product uses a simpler internal format.
Give each classification rule an owner, a version and examples at its boundary. When a category is split or an alias corrected, identify which existing outputs need recalculation. Preserve enough history to explain whether a reported movement reflects changed activity or changed labeling.
Review recurring correction work with the teams supplying the data. Prioritize the source problem that repeatedly creates ambiguity, then test the revised rule before broad application. Track unresolved cases and downstream effects alongside throughput so a faster import cannot hide growing repair work.