Non-ASCII in the future in the lexer

[Dom DiSc]

On Wednesday, 31 May 2023 at 06:23:43 UTC, Cecil Ward wrote:
>
>
> What do you think? It occurred to me that as the language 
> develops we are occasionally having discussions about new 
> keywords, or even changing them, for example: s/body/do/ some 
> while back.
>
> Unicode has been around for 30 years now and yet it is not 
> getting fully used in programming languages for example. We are 
> still stuck in our minds with ASCII only. Should we in future 
> start mining the riches of unicode when we make changes to the 
> grammar of programming languages (and other grammars)?

I'm fully with you, but the problem is not to have any Unicode 
symbols in the grammar as operators or delimiters or whatever. 
It's the input method. Most keyboards don't have them on the keys 
and any other method is awfully slow. Even a well-designed 
selector table is slow if it is needed often - and most editors 
are far from providing such.