How every number is measured
The claims on these pages are only as good as the way they were checked. Here is every method behind them, including where they fail.
Every headline number comes from a random sample or a held-out set, never from examples picked by hand.
The sample. Papers drawn at random, with fixed seeds, from all 13,324 papers listed in August 2026, in two draws of 100 and 300 with no overlap. 366 of them could be labelled and are scored here.
The labelling. Each paper was labelled by reading its PDF, the author block, footnotes and acknowledgements, on a sheet that showed nothing the pipeline had decided. This round was labelled by a separate model pass; a human re-labels a random subset, and the agreement rate will be reported here.
The scoring. Precision and recall, with Wilson 95% confidence intervals.
| Question | Precision | Recall |
|---|---|---|
| Any author affiliation in mainland China or Hong Kong | 99%116/117 · 95–100 | 97.5%116/119 · 93–99 |
| The paper has a funding statement | 92%125/136 | 94%125/133 |
| A Chinese funder supports the paper | 100%29/29 · 88–100 | 76%29/38 · 61–87 |
Where it fails, honestly. Funder names are fully right for only about half of funding mentions: the name is found, but its boundaries are often too long or too short. Chinese-funder recall depends on how much of the funder registry has been labelled, the 220 most frequent names out of about 5,800 so far. The three affiliation misses are two papers whose HTML version loses the author block, and one Chinese state research programme named without its location.
The data. Wikidata people with both an English and a Chinese name, from mainland China, Taiwan, Hong Kong, Singapore and Malaysia, 5,775 in the test half. The split is 50/50 by a hash of each person's ID: the dialect table was learned only from the build half, and the test half was never inspected.
The pairs. A same-person pair is a person's English name against their own characters. A different-person pair is that English name against another person's characters with the same surname. The chart below is recall on the held-out half: how often the same person is matched, by region.
Calibration. The weights come from pairs of records that share an email address, the same person, set against random pairs. Human review. In a review of 30 pairs the matcher had left unresolved, 20 were the same person, 10 were unclear, and none were different people. The grading is deliberately conservative: a false merge is worse than a missed one when the output feeds analysis of real people.
Confirmed
Needs a hard identifier, an email, ORCID or grant PI, or identical characters plus corroboration.
Probable
The ceiling for soft signals alone: name, co-authors, institutions and topic.
Unresolved
The evidence is not enough. Left for a human, never forced into a merge.
- Tests. 92 automated tests on invented papers, people and organisations, run on every push against Python 3.10 and 3.13.
- Privacy. No real researcher appears in any test or in the public repository. In the review app, researcher data never leaves the analyst's machine; it binds to localhost, and decisions are stored separately from machine output.
- Reproducibility. Samples are drawn with fixed seeds; scope definitions are stored separately from the evidence, so changing a definition needs no re-run.