UTF8 + SIMD = win

[jerro]

On Tuesday, 31 July 2012 at 19:41:02 UTC, Tobias Pankrath wrote:
> On Tuesday, 31 July 2012 at 19:28:03 UTC, Walter Bright wrote:
>> On 7/31/2012 5:24 AM, Jakob Ovrum wrote:
>>> On Tuesday, 31 July 2012 at 12:11:25 UTC, bearophile wrote:
>>>> Bernard Helyer:
>>>>
>>>>> Where is UTF-32 actually used?
>>>>
>>>> I think all std.algorithm and std.range yield UTF-32 dchars, 
>>>> when you give
>>>> them a string in input.
>>>>
>>>> Bye,
>>>> bearophile
>>>
>>> In addition, foreach over a string with a dchar loop variable 
>>> does implicit
>>> UTF-8 decoding.
>>>
>>
>> SIMD isn't going to speed things up at all for decoding one 
>> character. It is for transcoding a large array.
>
> You could decode them in advance.

The problem is you don't know how much you are going to need.
This would actually hurt performance in some cases.