Blog

Better UnArchiver

Why ZIP filenames break on Mac — and what “encoding override” actually does

Mail a ZIP from an older Windows toolchain and open it on macOS: folders become mojibake. Archive Utility assumes UTF-8. The ZIP format historically did not require a reliable filename charset field, so readers guess.

The ambiguity

General-purpose ZIP tools on Windows often stored non-ASCII names in a regional code page (for example GBK or Shift-JIS) without a flag modern Mac APIs understand as UTF-8. Once you decode with the wrong table, the original bytes are not recoverable from the already-wrong string — you need the archive again.

Detect, preview, then extract

Better UnArchiver treats encoding as part of inspect-first:

  1. Read raw filename bytes from central directory headers.
  2. Try candidate encodings (including GBK, Big5, Shift-JIS, EUC-KR) and show a preview.
  3. Let the user override when the automatic pick is wrong.
  4. Only then extract with the chosen mapping.
# Illustrative: score candidates by how many replacement characters appear
from collections import Counter

CANDIDATES = ["utf-8", "gbk", "big5", "shift_jis", "euc_kr"]

def preview(raw: bytes) -> list[tuple[str, str, int]]:
    rows = []
    for enc in CANDIDATES:
        try:
            text = raw.decode(enc)
        except UnicodeDecodeError:
            continue
        bad = text.count("\ufffd") + sum(1 for ch in text if ord(ch) < 32)
        rows.append((enc, text, bad))
    return sorted(rows, key=lambda r: r[2])

Production code also has to handle the UTF-8 bit when present, AppleDouble noise, and paths that attempt traversal. Those checks run before any write to disk.

What this does not fix

  • A ZIP that was re-saved after a wrong decode has already lost the original bytes.
  • RAR creation and RAR recovery-record repair are out of scope; RAR is extract-only in the current release.
  • Encoding overrides do not magically repair CRC-failed payloads; salvage writes readable entries to a new archive.

Related

Back to Blog