Why is BOM required to use unicode in tokens?

James Blachly james.blachly at gmail.com
Tue Sep 15 01:49:13 UTC 2020

I wish to write a function including ∂x and ∂y (these are trivial to 
type with appropriate keyboard shortcuts - alt+d on Mac), but without a 
unicode byte order mark at the beginning of the file, the lexer rejects 
the tokens.

It is not apparently easy to insert such marks (AFAICT no common tool 
does this specifically), while other languages work fine (i.e., accept 
unicode in their source) without it.

Is there a downside to at least presuming UTF-8?

More information about the Digitalmars-d-learn mailing list