UTF-8 Unicode operators vs digraphs

Walter Bright newshound2 at digitalmars.com
Wed Aug 12 19:50:18 UTC 2026


On 8/12/2026 5:30 AM, Dlighted wrote:
> - 𝔹 (Blackboard Bold Capital B): This belongs to the Unicode Mathematical 
> Alphanumeric Symbols block (U+1D539). These symbols are notorious for causing 
> tokenization inefficiencies. Because they look visually similar to a standard 
> Latin "B" to a human, but have completely different underlying byte codes, 
> tokenizers often fragment them into multiple awkward tokens.

Another one of my major gripes about Unicode - characters that render 
identically but are completely different.


More information about the Digitalmars-d mailing list