File Encoding Detector

Browsers don't ship chardet, and importing a large encoding-detection library (the JS port of ICU's CharsetDetector is ~200KB) for a few common cases is overkill. This tool covers the 95% case: BOMs, ASCII-only, valid UTF-8, and UTF-16 by null-byte pattern. For exotic Asian encodings (Shift_JIS, GB2312, EUC-KR) without a BOM, you'll need chardet — but this tool will tell you 'not UTF-8' so you know to look elsewhere.

Byte Order Marks are 2-4 byte prefixes that explicitly mark the encoding. UTF-8 BOM is EF BB BF (technically unnecessary, controversial — Microsoft adds them, Unix tools usually strip them). UTF-16/32 BOMs (FF FE etc.) are useful because they also signal endianness. If a file has a BOM, trust it absolutely.

Then the file is UTF-8 and the program reading it is not using UTF-8, which is a different problem with a different fix — a locale setting, an import dialogue, a database connection charset. Detection answers what the bytes are; it cannot make the next program agree. Check what the reader assumes before re-encoding anything.

Enough to contain the characters that distinguish one encoding from another. UTF-8 can be confirmed from a single multi-byte sequence, so even a short file is decisive when it has one. A pure-ASCII file is genuinely ambiguous at any length, because it is valid in almost every encoding — and a short line with one accented character is exactly the case where a detector has to guess.