lazy redux

[bearophile]

Andrei Alexandrescu:

> Should we sack lazy? I'd like it to have a reasonable replacement. Ideas 
> are welcome!

I am not yet able to suggest you a replacement.
This is a small post I have recently read about limits and problems of laziness in Scala, that looks similar to laziness in D:
http://pchiusano.blogspot.com/2009/05/optional-laziness-doesnt-quite-cut-it.html

I can also show you two usages of mine of lazy. I have used lazy in D1 for two main purposes:

The first problem come from translating Python code to D. In Python the 'or' operator is lazy, and it returns the first not false object it sees (in Python empty collections are false), few examples:

>>> 0 or 4
4
>>> [] or [1,2] or [3]
[1, 2]
>>> (5,) or [1,2]
(5,)
>>> x = 1
>>> def foo(): global x; x += 1
...
>>> [5] or foo()
[5]
>>> x
1
>>> [] or foo()
>>> x
2

It's easy to implement a n-way version of that, this is a reduced 2-way (not tested):

T1 lazyOr(T1, T2)(T1 x, lazy T2 y) {
    static assert(CastableType!(T1, T2), "...");
    if (boolean(x))
        return x;
    else
    	return y;
}


Another small problem of lazy in D is that the caller doesn't know that an expression will be used in a lazy way. This is sometimes handy, but also is not very explicit, so it can lead to troubles. A simile solution to this is to require the 'lazy' on the calling site too:

auto x = lazyOr(a, lazy b);

I don't like that a lot, but it's more explicit.


I have used lazy to implement a poor's man version of the array comps of Python, that I've now seen I can't live without in D. The starting code that I have worked on was from Henning Hasemann in 2007.

This is one of those versions, I use more complex versions too:

TA[] select(TA, TI, TC, TP)(lazy TA mapper,
                            ref TI iter1, TC items1,
                            lazy TP where) {
    ArrayBuilder!(TA) result;
    auto aux1 = iter1; // save original iteration variable

    static if (IsAA!(TC)) {
        foreach (k, v; items1) {
            iter1 = k;
            if (where())
                result ~= mapper();
        }
    } else {
        foreach (el; items1) {
            iter1 = el;
            if (where())
                result ~= mapper();
        }
    }

    iter1 = aux1; // restore original iteration variable
    return result.toarray;
}


A small usage example:

int i;
select(toString(i), i, [1,2,3,4,5,6,7], i % 2 == 0)
==> ["2", "4", "6"]


The disadvantages of that code are big:
- It looks like magic, mostly because of lazy.
- It needs an already defined loop variable (here 'i').
- Even the LDC compiler is not able to compile that code well, so it's not top efficiency, despite the usage of ArrayBuilder inside it.
- It syntax is quite less readable than the Python version:
[str(i) for i in [1,2,3,4,5,6,7] if i % 2 == 0]
- It can be used to produce an actual array only, it can't be used for lazy generators as in Python:
(str(i) for i in [1,2,3,4,5,6,7] if i % 2 == 0)
- It's not a built-in thing, some D programmers may not understand or like it.


But its advantages are bigger than those big disadvantages:
- For not-performance critical parts of the code it's fast enough. Where the profile tells me that some code is slow it's easy to lower the level of the code.
- I have understood only now the main advantage of array comps. Master chess players are able to play many games at the same time and they are able to memorize the configuration of the pieces on many chessboards. Newbie chess players aren't able to remember so many boards. Experiments have shown that if the pieces are put randomly on the board, then master players are able to remember about as many chess positions as newbies or just a little more. During true games masters are able to memorize several chessboards because they don't memorize the position of each piece, they divide the boards in pieces that have a semantic meaning, and then memorize those few chunks. Such 'chunking' is essential during their play too, they think mostly in terms of those chunks, those gestalts, and often not with the movements of single pieces. This chunking is useful because human brains aren't able to manage more than a handful of separated items when they think (about seven), but such items can be complex, they are chunks. Python list comps allow to cut a piece of a code and think of it as a single chunk, allowing me to program at a higher level and better. This is why Python3 has added two more kinds of comps, for sets and dicts:

a_set = {x for x in xrange(10) if x & 1}
a_dict = {x*x : x for x in xrange(100)}

Bye,
bearophile