A Chinese name engine, done properly
Match a person across the scripts and dialects a Chinese name travels through, and get back not a yes or no but a weight of evidence and a reason you can read.
Most tools treat a name as a string. They either merge two different people or split one person into many.
A Chinese name reaches an English-language database through a lossy pipeline. 王伟 and 王薇 both become "Wang Wei". One person can appear as 张家欣, "Jiaxin Zhang", "Chia-Hsin Chang" (Taiwan, Wade-Giles), "Cheung Ka Yan" (Hong Kong, Cantonese) or "Teo Kah Hin" (Singapore, Teochew). The surname order flips, and English given names get bolted on ("Jasper Lee Po Lung"). A tool that compares these as text will get it wrong in both directions.
Every name is parsed into all the readings it could have, and each comparison returns three things: a level (identical characters, a reading that fits, initials that fit, uncertain, partial, or a conflict); a weight of evidence, a log-likelihood ratio the identity matcher can add to other evidence; and a reason a reviewer can read. Three rules keep it honest.
- 1A match counts in proportion to how rare the name is. "Wang Wei" matching "Wang Wei" is weak evidence; a rare name is strong evidence.
- 2A mismatch counts in proportion to how reliable both forms are. Two Pinyin forms that differ are strong evidence against a match; a mismatch through an approximate romanisation is weak.
- 3A spelling the engine cannot read is never a mismatch. It is "uncertain", which means absence of evidence, not evidence of absence.
| System | Where | Example |
|---|---|---|
| Hanyu Pinyin | Mainland China | 张家欣 → Zhang Jiaxin |
| Wade-Giles | Taiwan, older literature | 张家欣 → Chang Chia-Hsin |
| Cantonese (HK government style) | Hong Kong, Macau | 鄧樹榮 → Tang Shu-wing |
| Hokkien, Teochew, Hakka | Singapore, Malaysia | 陈文庆 → Tan Boon Keng |
Cantonese readings come from the Unicode Han Database (Jyutping for 30,096 characters), converted to the Hong Kong government spellings and their common variants. The Singapore and Malaysia dialect table was learned from people with both an English and a Chinese name in Wikidata, then reviewed row by row.
Measured on the held-out half of the data: people whose dialect spellings the engine never saw while it was built. The chart is recall, the rate at which the same person is matched. Two additions moved these numbers: the dialect table cut Singapore and Malaysia misses from 12.5% and 15.2% to 5% or less, and the Cantonese readings raised Hong Kong matches from 13% to 37%.
- Stage names, pen names, married names and Malay transliterations are not recognised.
- The test set is Wikidata, which skews towards public figures and older generations.
- The dialect table was reviewed by a non-native speaker of those dialects.