Transience of .front in input vs. forward ranges

[Jonathan M Davis]

On Sunday, November 04, 2012 02:30:49 Era Scarecrow wrote:
>   From watching and gleaming from what I have so far, I can only
> think that transient should NOT be the default way that ranges
> work (as it causes too many problems); However transient should
> be allowed (and available) when possible.
> 
>   I can only think that perhaps using a template to determine if
> it's transient should be present, that way it's a simple flag to
> enable/disable and should propagate anywhere that that range was
> used.
> 
> struct Range(bool isTransient=false) {
>    static if (isTransient) {
>      //front and transient
>    } else {
>      //front and non-transient
>    }
> }
> 
>   Unless someone thinks this is a bad approach?

That's just trying to be able to tell a range whether it should be transient 
or not and doesn't really solve the problem. The problem is that algorithms in 
general assume that front is _not_ transient, and we need to either decide 
that algorithms are free to continue to assume that front is non-transient 
(meaning that you make front transient on your range at your own risk and no 
guarantees that any range-based functions will work with it), or we need to 
provide a way for range-based functions to know whether a particular range has 
a transient front or not. As such, I think that it comes down primarily to 
either

1. Just decide that all input ranges can have transient fronts and that no 
ranges greater than input ranges can have transient fronts. Then algorithms 
can infer transience from the type of range. This is how Andrei thinks that 
it's supposed to work but pretty much no one else does (if nothing else, 
because that idea was never made clear anywhere, and no one else inferred that 
from the definitions of input ranges).

2. Make it so that any range can have a transient front but provide a template 
for checking for it like we do with stuff like hasSlicing or length. Presumably 
that template would check that for something like like the existence of 
R.isTransient, and ranges with transient fronts would just declare an enum 
with that name. Then every range-based function which relied on front not 
being transient would have to check whether the range that it was given was 
transient or not or fail to compile if it is, and every wrapper range would 
have to propogate the isTransient member. Failure to do so in either case 
(which _would_ happen at least some of the time, at least outside of Phobos), 
would result in transient ranges silently doing bizarre things.

3. Make it so that ranges which can be transient are non-transient by default 
but provide a function to get at a transient version for speed (which was the 
fastRange proposal in this thread). The main problem here is that when the 
fast range gets wrapped, it's transient, and so anything using the wrapped 
range will be forced to use the transient version rather than using the non-
transient version and only using the transient version when it's asked for. 
So, I don't think that this is particularly viable.

4. Just decide that range-based algorithms can assume that front isn't ever 
transient. Any range types which make it transient do so at their own risk.

Honestly, I'd really like to go with #4 and make ByLine and ByChunk use 
opApply. I think that transience complicates things too much. Even just having 
input ranges have transient fronts really screws with things. Something as 
simple as

auto app = appender!E();
foreach(e; range)
    app.put(e);

is totally screwed by transience, and it's only operating on input ranges. I 
seriously question that a function like std.array.array can be implemented 
with a transient front. Without a way to explicitly copy front, you can't have 
front be transient and keep it like std.array.array does. And even if there 
_were_ a way to explictly copy the result of front, that would be horribly 
inefficient in the cases where front isn't transient, because it would force you 
to copy when it was completely unnecessary.

I honestly don't think that transience really works outside of some very 
specific use cases, and the cost of trying to support it will not be small. And 
given the issues that std.array.array has with it and the fact that 
std.array.array really shouldn't have to operate on forward ranges, I don't 
think that the idea of input ranges having transient fronts is really going to 
work, which would mean using something like isTransient, which has the same 
sort of problems as save tends to and is likely to be forgotten even more than 
save is. So, I really question that transience and ranges is really going to 
work at all. If we _do_ support it though, I think that we need to go the 
isTransient route.

- Jonathan M Davis