A problem with generators

[Pelle Månsson]

On 03/17/2010 10:26 PM, bearophile wrote:
> D currently has two ways to define a generator, the first is the older one with opApply (And opApplyReverse):
> int opApply(int delegate(ref Type [, ...]) dg);
>
>
> The syntax of opApply is terrible: hard to remember, bug-prone, noisy, intrusive. And this syntax doesn't even gain performance, because dmd is not very good at inlining here.
> Several times in the past I have said that a very good syntax for this is the Python one:
>
> def foo():
>    yield 5
>
> It's really easy to use and remember, not bug prone. This possible syntax is about 75 times better than the current opApply:
>
> yield(int) foo() {
>    yield 5;
> }
>
>
> That's sugar for something like:
>
> struct foo {
>      int opApply(int delegate(ref int) dg) {
>          int result;
>          int r = 5;
>          result = dg(r);
>          if (result)
>              goto END;
>        END:
>          return result;
>      }
> }
>
>
> Python syntax was good enough that C# has copied it.
>
>
> The other way to create a struct/class generator in D2 is with the Range protocol, with the methods that ask for the next item, etc. I have seen that such generators sometimes are harder to write, because you have to manage the state yourself, but they can be more efficient and they are more powerful.
>
> Both ways are useful, because they are useful for different situations. They are one the inverted-out version of the other.
>
>
> The implementations of generators in C# and Python (and D, but not Lua) suffer of a problem, that increases the computational complexity of the code when there is a nested generator.
>
> And both Python and C# have found similar solutions that improve both the syntax and the performance (both languages have not yet implemented it).
>
> Python version:
> http://python.org/dev/peps/pep-0380/
>
> With that this code:
>
> def foo():
>    for x in some_iterable:
>      yield x
>
>
> Becomes:
>
> def foo():
>    yield from some_iterable
>
>
> In C# it can be called "yield foreach":
>
> http://kirillosenkov.blogspot.com/2007/10/yield-foreach.html
>
> http://connect.microsoft.com/VisualStudio/feedback/details/256934/yield-return-to-also-yield-collections
>
> http://blogs.msdn.com/wesdyer/archive/2007/03/23/all-about-iterators.aspx
>
>
> As D2 will start coming out of its embryo state, it too will probably have to fece this problem with iterators. In the meantime I hope the opApply syntax will be replaced by something better (the yield I've shown).
>
> Bye,
> bearophile

While I think it should be done properly with ranges and not opApply, I 
strongly support this syntactic sugar.