[Issue 430] New: incorrect UTF-8 detection for drafted UTF-16/32 source files

[d-bugmail at puremagic.com]

http://d.puremagic.com/issues/show_bug.cgi?id=430

           Summary: incorrect UTF-8 detection for drafted UTF-16/32 source
                    files
           Product: D
           Version: 0.169
          Platform: PC
        OS/Version: Linux
            Status: NEW
          Keywords: rejects-valid
          Severity: normal
          Priority: P3
         Component: DMD
        AssignedTo: bugzilla at digitalmars.com
        ReportedBy: thomas-dloop at kuehne.cn


The current encoding detection for files without BOMs(module.c:490-522)
is broken for drafted cases.

---sample.d---
T x;

alias int T;
--------------

If T is replaced by a character >= 0x100 and if the file is saved as
UTF-16-BE, UTF-16-LE, UTF-32-BE or UTF-32-LE (all without BOM),
DMD-0.169 will erroneously treat the file as UTF-8.


--