Generality creep

Andrei Alexandrescu SeeWebsiteForEmail at erdani.org
Thu Mar 28 18:10:47 UTC 2019


On 3/28/19 1:49 PM, H. S. Teoh wrote:
> On Thu, Mar 28, 2019 at 05:33:33PM +0000, Luís Marques via Digitalmars-d wrote:
>> On Thursday, 28 March 2019 at 17:04:58 UTC, Andrei Alexandrescu wrote:
>>> The overall message is we got bogged down on the "wrong" side of
>>> generality - cross-cutting and nonscalable code additions to support
>>> unprincipled and low-impact corner cases.
>>
>> Having a document that formally specifies what a range is and what its
>> properties are might have helped with that and a lot more. It would
>> still help with all of the issues yet to come.
> 
> Yes, the range API needs a formal, precise spec that leaves no holes.
> Currently, the semantics of .save is one such issue.
> 
> Another issue is what we used to call "transience": is the return value
> of .front required to persist past the next invocation of popFront?
> std.stdio.File.byLine is a big example that does *not* respect this, and
> as a result several Phobos algorithms cannot be used with it without
> incurring buggy behaviour.

Yah, for such we need I think a more primitive notion of 
UnbufferedRange. An unbuffered range of T has only one API:

bool fetchNext(ref T);

Note that the user provides all state, which is interesting. The 
primitives fills the object and moves to the next one. Returns true on 
success. Returns persistently false at end of range even if called 
multiple times. An optional interface (for efficiency's sake) would be:

size_t fetchNextN(scope T[]);

which fills a full array and returns the number of elements filled.

These are trivially implementable from the outside for arrays etc.

Then we'd have input ranges, which have a buffer of at least one 
element, i.e. today's input ranges. Input ranges that are not forward 
ranges are liable to reuse their buffers, so after a call to front(), a 
call to popFront() may overwrite the current front. This is because by 
construction input ranges that are not forward ranges do not iterate 
objects in memory, but instead they transfer data from somewhere else 
into memory, chunkwise.

Forward ranges and above are guaranteed to walk real objects in memory 
and as such provide better guarantees on the persistence of their contents.

> OTOH, supporting transience means code like
> the following won't work:
> 
> 	auto e = r.front;
> 	r.popFront;
> 	if (r.front == e) ...
> 
> because e becomes invalid after .popFront.  Last time I checked, several
> Phobos algorithms freely use this kind of construct (saving the value of
> .front and checking it later), which means it doesn't work with
> transient ranges.
> 
> We need a clear decision as to whether transient ranges are supported.
> If not, std.stdio.File.byLine needs to be deprecated. Or if they should
> be supported, then the range spec should clearly state that it is
> illegal to save the value of .front and expect it to remain valid after
> .popFront is called. (Personally, I lean towards the latter option, but
> either way, this needs to be stated clearly and explicitly in the range
> spec.)

If we support any notion of input range, what follows is a sequence of 
forced moves: we must support iteration of objects that are not in 
memory, hence we need to support transiency of data.


More information about the Digitalmars-d mailing list