Super-dee-duper D features

[Andrei Alexandrescu (See Website For Email)]

kris wrote:
> Andrei Alexandrescu (See Website For Email) wrote:
>> Bill Baxter wrote:
>>
>>> kris wrote:
>>>
>>>> kris wrote:
>>>>
>>>>> Bill Baxter wrote:
>>>>>
>>>>>> Frits van Bommel wrote:
>>>>>>
>>>>>>> By the way, would the new loop syntax allow more than two 
>>>>>>> collections to be simultaneously iterated?
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Whoa!  I certainly hope so.  It hadn't even occurred to me that 
>>>>>> Andrei might mean this syntax can only be used for just two 
>>>>>> collections.  If that's the case then ... ick.
>>>>>>
>>>>>> --bb
>>>>>
>>>>>
>>>>>
>>>>> InterleavedIterator does multiple collections via using multiple 
>>>>> instances of InterleavedIterator. It's simple to use, and only 
>>>>> needs to be written once. Would be better to implement some basic 
>>>>> iterator needs than to introduce some tricky new syntax?
>>>>>
>>>>> - Kris
>>>>
>>>>
>>>>
>>>> Should have given an example. Simple case with 2 entities:
>>>>
>>>> auto two = InterleavedInterator (x, y);
>>>> foreach (x; two) ...
>>>>
>>>> more than 2:
>>>>
>>>> auto three = InterleavedInterator (two, z);
>>>> foreach (x; three) ...
>>>
>>>
>>> Should have also mentioned where one can find this mythical 
>>> InterleavedIterator.
>>
>>
>> The issue with such a multi-iterator is that it makes it easier to 
>> make errors, and harder to write efficient and correct code that's 
>> statically verifiable.
>>
>> I'm not sure when the interleaved iterator stops iterating, but there 
>> are two possibilities, none of which is satisfactory:
>>
>> 1. Stop after the shortest of the two collections is done. Then user 
>> code must query the state of the iterator after the loop to figure 
>> what extra work is to be done:
>>
>> auto two = InterleavedInterator (x, y);
>> foreach (x; two) { ... }
>> if (two.MoreData(0)) {
>>   auto back2one = two.Project(0); // fetch the first iterator
>>   foreach (x ; back2one) { ... }
>> } else if (two.moreData(1)) {
>>   ... same (or)deal ...
>> }
>>
>> This is way more work than there should.
>>
>> 2. Stop after the longest of the two collections is done. Then user 
>> code must ensure _at each step_ that both iterators have meaningful data:
>>
>> auto two = InterleavedInterator (x, y);
>> foreach (x; two) {
>>   if (two.HasData(0)) { ... }
>>   else { ... only the second iter has data ... }
>> }
>>
>> This is unclear, verbose, and probably suboptimal.
>>
>> The scoping of foreach links the scope of the variables with their 
>> validity range, which rules out a class of possible errors entirely:
>>
>> foreach (x ; c1) (y ; c2) (z ; c3) {
>>   ... x, y, z syntactically accessible _and_ valid ...
>> }
>> continue foreach (x, z) {
>>   ... x is both invalid _and_ syntactically inaccessible ...
>> }
>>
>> As I mentioned in a different post, the fact that there are 
>> combinatorial potential sub-foreach statements is a non-issue.
>>
>>
>> Andrei
> 
> If x, y, & z are of differing type, then I'd agree.

If they are of the same type and in an arbitrarily large numbers (x1, 
x2, x3...), we start talking about cutting through some sort of a matrix 
or manifold, which is an entirely different business.

Anyhow, I think it's clear by now that the language makes some idioms 
faster, and library iterators make some other idioms faster. Clearly 
library iterators are useful. The question is if the language-helped 
idioms are encountered often enough to justify the cognitive load of 
implementing them.

I'm biased by my own C++ codebase, which does a _ton_ of looping (linear 
algebra, neural nets, manifold learning...) I have a nice FOREACH(i, 0, 
n) macro that takes care very effectively of most loops, with proper 
type deduction (gotta love gcc's typeof), limit hoisting, you name it. 
(I've sat down and measured that it has no impact on the efficiency of 
the generated code, which is paramount.)

In contrast, the few places in which I had to use a straight for loop or 
maintain extra variables to do parallel iterations really make the thin 
facade break down as all of a sudden I need to fully explain myself to 
the compiler. All of the extra cases could be helped, and efficiently, 
by continue foreach (actually, to tell the truth, I could also use the 
foreach_reverse/continue foreach_reverse correspondent feature).


Andrei