Last week I published 109 Chinese surnames as open data: JSON and CSV, CC BY 4.0, one record per surname with pinyin, the era it took shape, its founding ancestor, its commanderies, its hall names, its population rank and a prose account of where it came from.

This week I re-released it as v2.1 because the dataset was citing the wrong source for 30 of its numbers.

No value changed. The ranks were already right, the headcounts were already there, and the 109 records are byte-identical apart from one new field. What changed was a sentence in the docs, and the sentence was wrong in a way that made every headcount in the file unreliable to quote. I think that's the more useful story than the dataset itself.

The check that found it

I was drafting a follow-up post about the awkward shape of this data — that only 30 of the 109 surnames have a national population rank, that everything below the official top 100 is somebody's estimate wearing a suit. Before writing it I did the obvious sanity check, the one you'd write in thirty seconds:

import json, urllib.request rows = json.load(urllib.request.urlopen( "https://naming-cn.pages.dev/data/surnames.json")) ranked = sorted((r for r in rows if r["rank_num"]), key=lambda r: r["rank_num"]) for a, b in zip(ranked, ranked[1:]): if b["population_est"] > a["population_est"]: print(f'{b["surname"]} #{b["rank_num"]}: {b["population_est"]:,}' f' > {a["surname"]} #{a["rank_num"]}: {a["population_est"]:,}')

If the ranking is a ranking, a surname ranked lower cannot have more people than the one above it. The loop should print nothing. It printed this:

林 #18: 14,200,000 > 何 #17: 13,000,000 高 #20: 14,500,000 > 罗 #19: 14,000,000 宋 #24: 11,200,000 > 谢 #23: 10,500,000 韩 #27: 9,800,000 > 许 #26: 9,000,000 冯 #29: 9,500,000 > 邓 #28: 9,200,000

Five inversions, all adjacent, all in the same stretch of the table. My first reaction was that I'd fat-fingered five extractions in a row.

It wasn't a typo

I opened each surname page against the JSON. 何 at 13 million and 林 at 14.2 million are both plausible numbers; so are 罗 and 高, 谢 and 宋. Every individual value looked reasonable in isolation. The only thing wrong was their relationship — and only in five places out of 29 adjacent pairs.

That pattern doesn't come from bad typing. Typing errors don't cluster into adjacent pairs and stay plausible. It comes from putting two different columns next to each other and calling them one column.

So I went back to the primary document: the Ministry of Public Security's 2020 national name report, which is the only official Chinese surname statistics at national scale, and which my docs credited for "population ranks and sizes".

What the report actually contains

Here's the part I should have checked before shipping v1.

The report's 百家姓 table ranks the top 100 surnames. It prints the name and the rank. It does not print a headcount. Not "sizes for the top 100" as my docs claimed — the column simply isn't there.

The only figures the report does print are in a separate section on common compound surnames: 欧阳 111.2万, 令狐 5.5万, 诸葛 4.8万, 皇甫 6.4万, 上官 8.8万, and seven more. Twelve of them. And when I compared those twelve against my data, they matched character for character, down to the 万 formatting.

So the file was not a fabrication. It was worse in a subtler way: 12 of the population figures genuinely come from the report, and the other 30 do not. The 30 belong to ranked single-character surnames — 王, 李, 张, 刘, 陈 — whose headcounts come from other surveys and estimates of varying vintage. Some of them are probably close to right. I no longer know which survey each one came from, because the trail goes through sources that all cite the same sentence.

And that sentence is everywhere. Search for any of these numbers and you'll find dozens of pages saying "according to the 2020 Ministry of Public Security report, 李 has 101 million people". They are all repeating an attribution, not a table. A citation repeated often enough starts to look like evidence, and I inherited it the same way everyone else did.

Provenance is not transitive. That's the whole lesson, and I paid for it with a version number.

Why the error was invisible until now

Because the wrong citation is harmless to almost every use of the data.

The ranks are correct. The headcounts are in the right order of magnitude. The only thing that's false is the claim that the two columns share a source — and that claim only matters if you (a) sort by population, or (b) quote a headcount as official. Publish only ranks, and nothing is wrong. Publish only headcounts, and nothing is wrong. Publish both with a single blanket citation, and you've created an error that no reader can see without cross-checking the report themselves.

That's the shape of most data-provenance bugs, I suspect. Not "the number is wrong" but "the number is right and the sentence about it is wrong", which is harder to notice and harder to care about.

The fix: one additive field

v2.1 adds population_source to every record. Four values:

Value Meaning Records
mps2020 figure printed verbatim in the 2020 report's compound-surname section 12
estimate some other statistic or estimate, vintage varies, not official 30
qualitative only the remark 人数稀少 (recorded as rare), no figure 18
"" no data at all 49

Additive only: every v2.0 field keeps its name, order and value, so nobody's parser breaks. If you want only what the report actually says:

import pandas as pd df = pd.read_csv("https://naming-cn.pages.dev/data/surnames.csv") official = df[df["population_source"] == "mps2020"] # 12 rows, quotable estimated = df[df["population_source"] == "estimate"] # 30 rows, cite the caveat

A field is better than a paragraph because a field joins. My old docs had a "sources" section at the bottom that said the ranks and sizes came from the 2020 report. It was prose, so it could be wrong for 30 of 42 records without anything failing. Now the provenance travels in the same row as the number, and the difference between official and not is a filter, not a footnote.

Make the invariant fail loudly

The other half of the fix is unglamorous: the sync script that mirrors the canonical files into the GitHub repo now enforces the semantics of the new field, and prints the inversions on every run.

population_source must be one of mps2020 / estimate / qualitative / "" mps2020 → must carry a figure, must NOT carry a rank (the report's top-100 table has no headcount column) estimate → must carry a figure qualitative→ must carry "人数稀少" and no number "" → must carry neither

And at the end of a successful sync:

OK 109 records · 30 with a 2020 rank · 42 with a population figure · 18 marked rare provenance: 12 mps2020 · 30 estimate · 18 qualitative · 49 none note: 5 adjacent inversion(s) when sorting by population_est — 林#18 > 何#17, 高#20 > 罗#19, 宋#24 > 谢#23, 韩#27 > 许#26, 冯#29 > 邓#28 sort on rank_num; use population_est for magnitude only

The five pairs are printed rather than suppressed, because they are a real property of the data and not a bug to hide. If a sixth ever appears, the numbers have drifted further apart than the dataset is willing to admit, and I want to see it on the next Monday's cron run rather than in a stranger's bug report.

Three things I'd tell anyone publishing a dataset

Ship the provenance in the row, not in a paragraph. A sources section is a promise; a column is a fact. The moment two numbers in the same table have different lineages, they need separate fields, even if that means an "obvious" column that's the same value 109 times.

Write a check that can fail. I read all 109 records while building this dataset and noticed nothing, because reading checks plausibility and plausibility was fine. One loop comparing adjacent rows found the problem in a second. Invariants are cheap; the only cost is admitting you can't tell by looking.

Go to the table, not the sentence about the table. I trusted the sentence, and the sentence was a summary written by someone who also trusted a sentence. The primary document took four minutes to find and settled it immediately.

The data

CC BY 4.0. Commercial use, modification and redistribution are all fine; attribution plus a licence link is the only condition.

Corrections are genuinely welcome — and if you know where those 30 headcounts really came from, I would rather have the citation than the number.