UTF-8 Unicode operators vs digraphs

Dlighted test at example.com
Wed Aug 12 12:30:40 UTC 2026


On Wednesday, 12 August 2026 at 11:52:44 UTC, Timon Gehr wrote:
> Instead of discussing what is interesting about the language, a 
> good chunk of the comments are just low-effort 
> anti-Unicode-syntax posts. It's not considered sufficiently 
> "normal" and people like writing snarky comments, so you always 
> get a lot of spam whenever you are trying to share something.

Read this post and reflect.

> I think `∧` and `∨` I think are not great choices for `&&` and 
> `||`, because `&&` and `||` are short-circuiting operators. 
> Then `and` and `or` don't match either, for the same reason 
> (though I guess Python does this anyway). One alternative I 
> have seen that actually makes sense is `and then` and `or 
> else`, but it's a bit verbose.

There's a long list of languages that use "and" and "or".
Pascal, Ada, Fortran, Lua, Python, PHP, R and probably others.
Ruby and Perl have them but with different precedence.

>OpenAI ignores 𝔹 when reading HTML pages, so GPT is actually not 
>able to read documentation if it is written with Unicode.

Gemini says its a problem with the letter you chose and not with 
Unicode in general:

Modern AI models (like ChatGPT) can read both symbols, but they 
process them through a tokenizer that breaks text down into bytes 
or smaller subword chunks.
- ∨ (Logical OR / Down Tack): This is a standard 
mathematical/logical symbol present in Unicode (U+2228). Because 
it is frequently used in math, logic, and programming datasets 
online, tokenizers often handle it gracefully, sometimes mapping 
it directly or splitting it efficiently.
- 𝔹 (Blackboard Bold Capital B): This belongs to the Unicode 
Mathematical Alphanumeric Symbols block (U+1D539). These symbols 
are notorious for causing tokenization inefficiencies. Because 
they look visually similar to a standard Latin "B" to a human, 
but have completely different underlying byte codes, tokenizers 
often fragment them into multiple awkward tokens.


More information about the Digitalmars-d mailing list