byChunk odd behavior?

[Taylor Hillegeist via Digitalmars-d-learn]

On Tuesday, 22 March 2016 at 07:17:41 UTC, Hanh wrote:
> Hi all,
>
> I'm trying to process a rather large file as an InputRange and 
> run into something strange with byChunk / take.
>
> void test() {
> 	auto file = new File("test.txt");
> 	auto input = file.byChunk(2).joiner;
> 	input.take(3).array;
> 	foreach (char c; input) {
> 		writeln(c);
> 	}
> }
>
> Let's say test.txt contains "123456".
>
> The output will be
> 3
> 4
> 5
> 6
>
> The "take" consumed one chunk from the file, but if I increase 
> the chunk size to 4, then it won't.
>
> It looks like if "take" spans two chunks, it affects the input 
> range otherwise it doesn't.
>
> Actually, what is the easiest way to read a large file as a 
> stream? My file contains a bunch of serialized messages of 
> variable length.
>
> Thanks,
> --h

I dont know if this helps, but it looks like since take three 
doesn't consume the chunk it is not removed from the range.

import std.stdio;
import std.algorithm;
import std.range;

void main() {
	auto file = stdin;
	auto input = file.byChunk(2).joiner;
	
	foreach (char c; input.take(3).array) {
		writeln(c);
	}
	
	foreach (char c; input) {
		writeln(c);
	}
}

Produces:
1
2
3 < Got data but didn't eat the chunk.
3
4
5
6