Phobos usability with text files

[Andrei Alexandrescu]

On 12/26/10 11:53 AM, Michel Fortin wrote:
> On 2010-12-26 12:13:41 -0500, Andrei Alexandrescu
>> Generally I think buffer reuse in byLine() is too valuable to let go.
>
> I also agree it's wasteful.
>
> But I think bearophile's experiment has illustrated two noteworthy
> problems. The first issue is that calling filter! on the
> already-consumed result of byLine() gives you a seg fault. I reproduced
> this locally, but haven't pinpointed the problem.

That should be filed as a Phobos bug. Could you please do the honors?

> The second one is this:
>
> array(file.byline())
>
> which gives a wrong result because of the buffer reuse. Either it should
> not compile or it should idup every line (both of which are not entirely
> satisfactory, but they're better than getting wrong results).

I agree it would be great if that meaningless expression wouldn't compile.

> I think a range should be able to express if the value can be reused or
> not. If the value can't be reused, then the algorithm should either not
> instantiate, or in some cases it might create a copy.

I thought a couple of days about this. There's a host of desirable 
properties that could ideally be expressed during compilation. My 
experience is that inevitably they add complexity to the range 
definition. For example, we could define a trait like this:

template iteratesDistinctObjects(R) if (isInputRange!R) {
     enum iteratesDistinctObjects = true;
}

That would cover the common case, then the ByLine range would say:

template iteratesDistinctObjects(R) if (is(R == ByLine)) {
     enum iteratesDistinctObjects = false;
}

That means all existing ranges and algorithms need to be reviewed to 
take that property into account. The complexity of the library rises.

Even if we cover that, there are many properties like that, and user may 
still define their own ranges, algorithms, or plain old code that do the 
wrong thing in one way or another. So it all becomes a matter of 
deciding at which point we can reasonably ask the library user to Read 
The Fine Manual. I'm not sure where the decision about this particular 
issue should be.

By the way, a simple way to define a range that issues one new string 
per line is:

auto r = map!(to!string)(f.byLine());


Andrei