Prevent opening binary/other garbage files

[Adam D. Ruppe]

On Monday, 1 October 2018 at 15:21:24 UTC, helxi wrote:
> I tried out https://dlang.org/library/std/utf/validate.html 
> before manually checking for encoding myself so I ended up with 
> the code below. I was fairly surprised that "*.o" (object) 
> files are UTF encoded! Is it normal?

Yes. Any random collection of bytes <= 127 is valid utf-8. Lines 
will read until it sees a byte 10, and cut off from there.

Quite a few file formats have a 10 early on to detect text/binary 
transmission corruption, but even if they don't, it is a fairly 
common byte to see before too long and that cuts off your scan 
for later bytes.


You really are better off looking for those <32 bytes like I 
described earlier - a .o file will likely have some 1's and 3's 
early on which that will quickly detect, but those will also pass 
the validate test.