UTF-8 Unicode operators vs digraphs
Walter Bright
newshound2 at digitalmars.com
Wed Aug 12 19:50:18 UTC 2026
On 8/12/2026 5:30 AM, Dlighted wrote:
> - 𝔹 (Blackboard Bold Capital B): This belongs to the Unicode Mathematical
> Alphanumeric Symbols block (U+1D539). These symbols are notorious for causing
> tokenization inefficiencies. Because they look visually similar to a standard
> Latin "B" to a human, but have completely different underlying byte codes,
> tokenizers often fragment them into multiple awkward tokens.
Another one of my major gripes about Unicode - characters that render
identically but are completely different.
More information about the Digitalmars-d
mailing list