byChunk odd behavior?

[Ali Çehreli via Digitalmars-d-learn]

On 03/22/2016 12:17 AM, Hanh wrote:
 > Hi all,
 >
 > I'm trying to process a rather large file as an InputRange and run into
 > something strange with byChunk / take.
 >
 > void test() {
 >      auto file = new File("test.txt");
 >      auto input = file.byChunk(2).joiner;
 >      input.take(3).array;
 >      foreach (char c; input) {
 >          writeln(c);
 >      }
 > }
 >
 > Let's say test.txt contains "123456".
 >
 > The output will be
 > 3
 > 4
 > 5
 > 6
 >
 > The "take" consumed one chunk from the file, but if I increase the chunk
 > size to 4, then it won't.

I don't understand the issue fully but byChunk() will treat every 
character in the file. So, even the newline character(s) are considered.

 > Actually, what is the easiest way to read a large file as a stream? My
 > file contains a bunch of serialized messages of variable length.

If it's a text file I think I would start with File.byLine (or 
byLineCopy). Then it depends on how the messages are layed out. One per 
line? Do you know the size at the start? etc.

Alternatively, use (or examine) one of the great D serialization modules 
out there. :)

(We already need something like this in the standard library, which I 
think some people are already working on.)

Ali