RFC: std.json sucessor

[via Digitalmars-d]

On Monday, 25 August 2014 at 21:53:50 UTC, Ola Fosheim Grøstad 
wrote:
> I presume you can load 16 bytes and do BITWISE-AND on the MSB, 
> then match against string-end and carefully use this to boost 
> performance of simultanous UTF validation, escape-scanning, and 
> string-end scan. A bit tricky, of course.

I think it is doable and worth it…

https://software.intel.com/sites/landingpage/IntrinsicsGuide/

e.g.:

__mmask16 _mm_cmpeq_epu8_mask (__m128i a, __m128i b)
__mmask32 _mm256_cmpeq_epu8_mask (__m256i a, __m256i b)
__mmask64 _mm512_cmpeq_epu8_mask (__m512i a, __m512i b)
__mmask16 _mm_test_epi8_mask (__m128i a, __m128i b)
etc.

So you can:

1. preload registers with "\\\\\\\\…" ,  "\"\"…"  and "\0\0\0…"
2. then compare signed/unsigned/equal whatever.
3. then load 16,32 or 64 bytes of data and stream until the masks 
trigger
4. tests masks
5. resolve any potential issues, goto 3