Tricky semantics of ranges & potentially numerous Phobos bugs

[Jonathan M Davis]

On Wednesday, October 17, 2012 00:56:30 Jens Mueller wrote:
> I've not really looked at byLine. Why do you have to do an allocation
> for each line?
> byChunk seems to not suffer from this problem (see
> http://d.puremagic.com/issues/show_bug.cgi?id=8085). Does this mean
> byChunk is less efficient?
> 
> I want to add that if byLine only implements opApply, then there should
> be some adapter that turns anything which has an opApply into a range.
> Then you have to do: byLine().adapter().joiner().
> 
> And I also think that the documentation needs to clarify whether
> popFront may invalidate t, if t was obtained before via auto t =
> r.front.

ByLine and ByChunk are in the same boat. If they want to function sanely as 
ranges, then they need to allocate a new buffer for every line/chunk. Neither 
of them do that, because that's inefficient in many cases (particularly when you 
don't need to keep a particular line/chunk around for the next iteration). 
Rather, they reuse their buffers. That means that when popFront is called, the 
previous result of front gets overwritten, which screws over a lot of range-
based algorithms. It's normally perfectly safe to assume that the result of 
front stays valid after popFront is called, but that's not the case with 
ByLine or ByChunk, which is what makes them so horrible as ranges.

If they use opApply, they still have the same problem insomuch as anyone using 
them has to understand that they reuse their buffers and that they must 
therefore dup the buffers if they want to keep anything around after an 
iteration, but then at least they won't be being passed to range-based 
functions where they often won't work and will end up with weird, confusing 
results (e.g. try using std.array.array on ByLine). Providing a means of 
getting a range from them is fine, but it needs to allocate a new buffer for 
each element in the range, or there _will_ be problems.

- Jonathan M Davis