Fix Phobos dependencies on autodecoding

[Gregor Mückl]

On Friday, 16 August 2019 at 21:05:44 UTC, Walter Bright wrote:
> On 8/16/2019 9:32 AM, xenon325 wrote:
>> On Thursday, 15 August 2019 at 22:23:13 UTC, Walter Bright 
>> wrote:
>>> And yet somehow people manage to read printed material 
>>> without all these problems.
>> 
>> If same glyphs had same codes, what will you do with these:
>> 
>> 1) Sort string.
>> 
>> In my phone's contact lists there are entries in russian, in 
>> english and mixed.
>> Now they are sorted as:
>> A (latin), B (latin), C, А (ru), Б, В (ru).
>> Wich is pretty easy to search/navigate.
>
> Except that there's no guarantee that whoever entered the data 
> used the right code point.

Depends. On smartphones, switching the keyboard language is easy 
(just a swipe on Android), so users that are regularly 
multilingual should be fine there. Windows also offers keyboard 
layout switching on the fly with an awkward keyboard shortcut, 
but it is pretty well hidden. So again, users that are 
multilingual in their daily routines should really be fine.

But taking a step back and trying to take a bird's eye view on 
this discussion, it becomes clear to me that the argument could 
be solved if there was a clear separation of text representations 
for processing (sorting, spell checking, whatever other NLP you 
can think of) and a completely seperate one for display. The 
transformation to the later would naturally be lossy and not 
perfectly reversible. The funny thing about that part is that 
text rendering with OpenType fonts is *already* doing exactly 
this transformation to derive the font specific glyph indices 
from the text. But all the bells and whistles in Unicode blur 
this boundary way too much. And this is what we are getting hung 
up over, I think.

Man, we really managed to go off track in this thread, didn't we? 
;)