If you have ever downloaded a ZIP file from a Japanese website and extracted it to find filenames like 縺薙s縺ォ縺。縺ッ or ƒeƒXƒg.txt, you have met mojibake — garbled text caused by decoding bytes with the wrong character encoding.
This is not a niche problem. Japan has a huge culture of self-published software, games, music and documents distributed as ZIP files, and most of them are created on Japanese Windows machines. The moment those archives are opened on a PC with a different system locale, the filenames fall apart. I ran into this often enough that I built the fix into my own archiver, Hodozip. This post explains the root cause and the detection approach.
ZIP filenames are bytes, not text
The ZIP format dates back to 1989, and its filename field was designed in a world of single-byte code pages. The original spec says filenames are encoded in IBM Code Page 437. A later extension added a flag — bit 11 of the general purpose bit flag, the "EFS" (UTF-8) flag — that marks a filename as UTF-8.
The catch: most archiving tools on Windows historically wrote filenames in the machine's local ANSI code page and did not set the UTF-8 flag:
- Japanese Windows → CP932 (Shift_JIS)
- Simplified Chinese Windows → GBK (CP936)
- Korean Windows → CP949
- Western Windows → CP1252
And most extraction tools, when the UTF-8 flag is absent, decode the filename bytes using their own machine's local code page. A CP932 byte sequence decoded as CP1252 (or vice versa) produces exactly the garbage shown above. Neither side did anything "wrong" by its own local rules — the archive simply doesn't carry the information needed to decode it correctly... except that it kind of does, as we'll see.
Why "just use the system locale" fails
Windows' own Explorer and many popular archivers tie filename decoding to the system locale. That works until you cross locales, which on the internet is immediately. Changing your system locale to match every archive's origin is not a real option — it requires a reboot, affects the whole OS, and you can only have one locale at a time while archives come from everywhere.
The fix: try the candidates, score them, pick the winner
The encoding isn't declared, but the bytes themselves are evidence. A filename encoded in CP932 has statistical structure: valid two-byte sequences, kana and kanji in plausible positions. Decoded with the wrong code page, you get control characters, stray symbols, half-width katakana runs, or replacement characters.
So Hodozip treats it as a small classification problem, per filename:
- If the UTF-8 flag is set, or the bytes are valid UTF-8 that also looks like real text, use UTF-8.
- Otherwise, decode the bytes with each candidate code page (CP932, GBK, and a few others).
- Score each result:
- Hard-invalid results (decode errors, control characters, replacement chars) score very low.
- Characters typical of the candidate language (e.g. hiragana/katakana for CP932, hanzi in common ranges for GBK) raise the score.
- Output that is technically valid but implausible as a filename scores in between.
- Extract using the winning decoding.
Scoring per filename rather than per archive matters: real-world archives occasionally mix files added by different tools on different machines.
This doesn't need machine learning or a dictionary — a handful of heuristics resolves the overwhelmingly common cases: a Japanese archive on an English PC, a Chinese archive on a Japanese PC, and so on. The ambiguous cases (very short filenames, pure ASCII) are harmless, because ASCII decodes identically in all candidates anyway.
Where this shows up in practice
- Downloading a free game or tool from a Japanese developer's site
- Receiving work files from a Chinese or Japanese client
- Opening old backup archives created on a PC with a different locale
In all of these, the "garbled name" problem used to mean renaming files by hand after extraction. With detection, the names just come out right.
Try it / read the code
Hodozip is a free, open-source (MIT) archiver for Windows 10/11. Encoding detection is on by default — double-click an archive and it extracts in place with correct filenames. It also handles batch extraction, 13 extraction formats, and ZIP/7z creation.
- Website: https://ren9618.github.io/Hodozip/
- Source: https://github.com/Ren9618/Hodozip
If you maintain an archiver or a library that touches ZIP files, the detection approach is easy to adapt: candidates + validity checks + plausibility scoring. And if you ship ZIP files to an international audience, please set the UTF-8 flag — future users will thank you.

