Lots of low hanging fruit in Phobos

[Nick Sabalausky]

On 3/7/2014 9:15 PM, Adam D. Ruppe wrote:
> On Saturday, 8 March 2014 at 01:10:38 UTC, H. S. Teoh wrote:
>> Having a way to auto-generate input range boilerplate, though, would
>> be really, *really* nice.
>
> Eh, I don't think it is a big deal and would be fairly limited compared
> to the current setup. If you use a fiber or state variable or something
> for the yield this yield that trick, how do you go backward? Random access?
>

Oh it would definitely be for nothing more advanced than a forward 
range. It's a coroutine, ie a generator function, so the elements are 
determined by execution flow. Since execution can't go in reverse or 
random-access, anything beyond forward range is ruled out. But you knew 
that :)

Forward range should be possible though (at least for pure-ish 
coroutines anyway), since all you'd have to do is clone the coroutine's 
internal state structure (which it must already have anyway, since you 
can't have coroutines without it).

While it couldn't be used for bidirectional or random-access ranges, it 
would still be a big help for generators - ie the whole point of 
coroutines in the first place. Other languages have coroutines - we'd 
have coroutines that can be used as ranges :)


> I think the best the yield stuff can do is maybe forward range and maybe
> infinite (probably with an annotation of some sort, since otherwise, the
> infiniteness wouldn't be obvious at compile time).
>
>
> So the best we're looking to automate is input or perhaps forward
> ranges. And how hard are these really to write?
>
> yield query(string q) {
>     auto result = c_query(toStringz(q));
>     while(!HasRow(result))
>        yield GetNextRow(result);
> }
>
> OK, that is kinda nice, but, is the status quo so bad? (BTW the reason I
> went with some kind of C database api here is everything else I could
> think of are actually pretty short when using std.algorithm functions to
> help define them.)
>
> struct query {
>      private Result result;
>      this(string q) {
>           result = c_query(toStringz(q));
>           if(!empty) popFront();
>      }
>
>      Row front;
>      @property bool empty() { return HasRow(result); }
>      void popFront() in { assert(!empty); } body {
>           front = GetNextRow(result);
>      }
> }
>
>
> It is certainly a bit longer, but it isn't that bad, and is easily
> extended to other range capabilities.
>
>
> Translating recursive iteration to a range does take a bit more, you
> need to track your local variables and put them in a stack of your own,
> but even that isn't too hard (though a bit wordier).
>
>
> I guess the whole yield thing can be kinda nice, I'm just not sure it is
> that big of a win given its other limitations compared to full ranges.

I dunno, what you wrote sounds to me like a pretty convincing argument 
in favor of coroutine ranges. ;)

Keep in mind, making *all* ranges easier to write isn't the charter 
here, and it doesn't need to be: Bidirectional and random-access ranges 
ARE more complicated than generators, so I think they're already just 
about as easy to write as we *can* make them.

Instead, the problem we're looking at solving here is exactly what 
you've described above: Making generators in D is more complicated and 
boilerplate-y than it really needs to be (unless you want to give up on 
ranges and use opApply - which still isn't particularly great without 
that mixin you suggested a couple years ago to cover up the return value 
ugliness).

Generators may be a subset of ranges, but they're an important subset. 
Unfortunately, languages like C# make us look bad when it comes to 
generators.

What really itches at me is that we're so tantalizingly close with 
opApply. The only real problem with it (aside from the return value 
system being kinda awkward compared to a "yield" statement) is that it 
can't be used as a range. And it can't be converted to a range without 
using Phobos fibers which imposes too much of an overhead to be a 
one-size-fits-all technique for generators.