DMD 0.170 release (foreach_reverse)

[Bill Baxter]

Chris Nicholson-Sauls wrote:
> Bill Baxter wrote:

>> And  'foreach(i; list.reversed)' is pretty clear as well.
> 
> Sure, and primitive arrays already have the .reverse property; however, 
> that means extra temporaries that hit performance and memory usage.  

That's why I said 'reversed' with a -d, which could be a property of arrays
that returns an opApply-style delegate (and could be implemented by 
classes and structs that care to as well).

> Both these hits are canceled out by adding foreach_reverse.  

Well, technically both of those hits are canceled out by a special case 
optimization in the compiler that checks if the argument of 
foreach_reverse is an array.  Having a foreach_reverse keyword just 
makes that optimization opportunity easier to spot.  The same 
optimization could be applied using 'foreach' by noticing that the 
argument of the foreach is a particular property of an array (.reversed 
.reverse_iter .opApplyReverse ... whatever).

 > The presence of opApplyReverse is primarily for completeness,

If you want completeness then you'll need to add foreach_inorder, 
foreach_preorder, foreach_postorder, foreach_randomized, 
foreach_every_other_one ... etc.

> and for generalization within templates.

Yes that was something Walter said, but I still don't think I've seen a 
concrete example of how this makes generic templates easier.  If I'm 
going to write a generic template that processes some items, I would 
think the best way to make it general (with the delegate-style iterator 
paradigm) would be to have the user pass in whatever delegate they want 
for iterating in whatever order they want.

I guess if you want to iterate backwards over items in a template it's 
useful, but that's more of a specialization than a generalization.  And 
that case could be handled just as well by unilaterally declaring 
opApplyReverse to be the "official name" of the reverse iterator method, 
and adding a corresponding opApplyReverse property to arrays.  Similarly 
in C++, the names "iterator" and "reverse_iterator" are just 
conventions.  You don't have to name your class's iterator that, but you 
should, because it will make your class more compatible with other 
people's code, and will be more readable to others.


>>   "Special cases aren't special enough to break the rules."
>>
>> As I understand it, the only real reason for foreach_reverse, when it 
>> comes down to it, is because of a strong desire to have a special case 
>> fast reverse iteration for arrays and strings.  But it really should 
>> be possible to hide that special case from the user and just detect 
>> some particular property e.g. "some_array.reversed" in a foreach 
>> statement. It may be a little more work (and maybe you just punt on 
>> the optimization till 2.0), but in the long run, the language and all 
>> it's users will benefit from removing that special case.
>>
>> --bb
> 
> Mostly agreeable.  However, neither does the language nor its users 
> suffer from having the foreach_reverse statement.

The price of a poorly thought-out feature in a language can have 
repercussions beyond how it affects one's day to day writing of code.

Already there have been a few people here who looked at foreach_reverse 
and said "if this is representative of D, then I'll find another 
alternative, thank you".  If the community shrinks, or fails to grow as 
it could, then that certainly impacts all D users.  It's all about the 
network effect.  I can say for myself when I saw the foreach_reverse 
addition it seriously made me consider whether there was any hope for D 
long term.  If this is the kind of design decision being made today, 
then what hope is there for the long term?  There's no Bell Labs or Sun 
Microsystems backing it, so it can't just power its way through bad 
design.  It's got to compete on quality.

As another example of the importance of intangibles in the network 
effect, folks here have repeatedly mentioned Nemerle's syntax extension 
/ lisp-like macro system.  The fact that people spoke highly of it here 
got me curious enough to go take a look at it.  I gave up on it after I 
saw it was a .NET thing, but still it was the network effect trying to 
take hold of me, and it probably would have kept me if I were in the 
target audience for Nemerle.

And I can say that personally I haven't done much evangelizing of D 
precisely because of issues like this.  One of the main attractions of D 
is the promise of a C++ designed properly.  That's a compelling story. 
If I could genuinely tell people "check out D -- it's C++ designed 
properly" then I would.  But right now, although D fixes many design 
problems and inconsistencies in C++, it introduces many others. 
foreach_reverse is definitely one such case of taking a step backwards 
from C++ in terms of design.

A good yardstick for how foreach_reverse -- or any such design issue -- 
may impact the network effect could be to poll the top contributors to 
D.  I think we can agree that these people are good for D, and it would 
be good for D to attract more people like them.  What do these MVPs 
think of the feature?  If it by-in-large turns them off, then probably 
it's a bad idea.  If it's 50/50 or greater then, fine, maybe we should 
have more features like foreach_reverse.

The bottom line is there aren't that many markets that really need the 
speed/efficiency of C++ any more.  Games, 3D software, multimedia, 
numerical and scientific computing, operating systems, 
databases/datamining, embedded systems.  That's about all I can think 
of.  Python and Ruby are probably fast enough for 80% of all 
applications, and Java is fast enough for %50 of what's left.  So D 
needs some powerful network effect going for it in order to avoid just 
withering away.  It needs to soak up as much of that last 10% as 
possible.  How?  You need people to love the language.  You need people 
to want to gush about it and love it so much that they're driven to 
write crazy stuff like this about it:
     http://poignantguide.net/ruby/chapter-2.html#section3
People don't gush about Visual Basic.  It gets the job done, but it's 
ugly, inconsistent, and idiosyncratic.  They gush about Ruby.  Why? What 
I keep hearing is things like "it's beautiful" "it's elegant" "it's 
expressive".  All that says to me a good, clean, concise, and consistent 
design.

This iterator stuff has got to be done right if iterators are to become 
a truly core concept in D.  If they are perceived as clunky or limited 
or inelegant, it will hurt D.

--bb