Parsing a UTF-16LE file line by line, BUG?
Nestor via Digitalmars-d-learn
digitalmars-d-learn at puremagic.com
Sat Jan 28 07:40:24 PST 2017
On Friday, 27 January 2017 at 04:26:31 UTC, Era Scarecrow wrote:
> Skipping the BOM is just a matter of skipping the first two
> bytes identifying it...
AFAIK in some cases the BOM takes up to 4 bytes (FOR UTF-32), so
when input encoding is unknown one must perform some kind of
detection in order to apply the correct transcoding later. I
thought by now dmd had this functionality built-in and exposed,
since the compiler itself seems to do it for source code units.
More information about the Digitalmars-d-learn
mailing list