Of possible interest: fast UTF8 validation

Wed May 16 13:54:05 UTC 2018

On 05/16/2018 08:47 AM, Ethan Watson wrote:
> On Wednesday, 16 May 2018 at 11:18:54 UTC, Andrei Alexandrescu wrote:
>> https://www.reddit.com/r/programming/comments/8js69n/validating_utf8_strings_using_as_little_as_07/ 
>>
> 
> I re-implemented some common string functionality at Remedy using SSE 
> 4.2 instructions. Pretty handy. Except we had to turn that code off for 
> released products since nowhere near enough people are running SSE 4.2 
> capable hardware.

Is it workable to have a runtime-initialized flag that controls using 
SSE vs. conservative?

> The code linked doesn't seem to use any instructions newer than SSE2, so 
> it's perfectly safe to run on any x64 processor. Could probably be sped 
> up with newer SSE instructions if you're only ever running internally on 
> hardware you control.

Even better!

Contributions would be very welcome.

Andrei