eating inout dogfood

[Steven Schveighoffer]

Recently, inout has become significantly useful in dmd.  The latest  
incarnation of the compiler in git has fixed all the previous issues with  
inout (special thanks to Kenji Hara for creating the related pull  
requests!)

Since I was a main designer of the inout system, I figured I should try  
and eat my own dogfood in terms of figuring out how well inout works for  
my main D side project -- dcollections.

So I went about modifying all of dcollections to be inout and const  
compliant.

The result was quite pleasant.  By applying inout and const to all the  
appropriate members, dcollections quickly became const aware, and I didn't  
have to add a single overload to do it!  There are a few issues with inout  
that I realize really do still need to be addressed.

For instance, we really do need a way to apply tail-const/tail-inout to  
custom structures.  Currently, any inout functions that return ranges or  
cursors return a fully-inout range or cursor.  This means, getting a range  
on a const container returns a range which cannot be iterated (you can  
still get the front and rear elements, call save, and other const-aware  
members).

So how do we fix this? One might think "why not just return an iterator  
with a tail-inout pointer?"  However, that doesn't work.  You cannot have  
a field that is inout, because inout is a temporary condition, only  
applicable to stack data.  For more details on this, see  
http://d.puremagic.com/issues/show_bug.cgi?id=6770

Not only that, but the issue is really the implicit casting.  Assuming the  
field-inout restriction didn't exist, we might define a simple cursor like  
this (for purposes of demonstration, I'm going to use a range over  
contiguous memory, ignoring the fact that slices already provide this):

struct cursor(T)
{
    T* node;
    void popFront() {node++;}
    ... // usual suspects front, empty, etc
}

This seems like cursor!(inout(V)) might work (V is the element type of the  
container) as the return type for inout functions.  However, one of the  
major requirements of inout is that it correctly cast back to the  
constancy of the container.

So this means cursor!(inout(V)) must cast to cursor!(const(V)).  However,  
as we all know, templates instantiated with different types where the  
types implicitly cast, do not allow implicit casting of the template.   
This is for good reason:

1. the representation might be different.  For example, int casts  
implicitly to long, but some struct S!int might not implicitly cast to  
S!long because S!long likely has a larger footprint.
2. the template might actually be completely different based on the  
parameters.  For example, you can use a static if to change the layout or  
functions depending on if the type is const or not.

I think this problem needs solving.  It would greatly improve the D story  
as a language where one can use superpowered generic programming to solve  
many problems that other languages need language modifications to solve.   
I have some ideas, but I wanted to let other people bring any ideas they  
might have first, as Walter has not been receptive to my attempts at  
solving this problem in the past.


============================
Another interesting "problem" I had while doing inout is I had many  
functions like this:

cursor elemAt(V v)
{
     cursor it;
     it.position = _hash.find(v);
     it._empty = it.position is _hash.end;
     return it;
}

But what I discovered (quite rapidly) is just applying inout to this  
function doesn't work.  When one is making a function that returns an  
inout aggregate, constructing that aggregate is almost required to be in  
the return expression.  So the above becomes:

inout(cursor) elemAt(const(V) v) inout
{
     auto pos = _hash.find(v);
     return inout(cursor)(pos, pos is _hash.end);
}

Although I think the latter is technically cleaner, it does bring up an  
interesting issue with regards to inout.  In some cases, applying inout is  
a no-brainer.  You just slap inout on the parameters and return values,  
and things just *work*.  However, in many cases, a redesign of the  
function is necessary.  This is something to keep in mind when promoting  
inout as a wonderful tool to bring const-awareness to existing code.


=============================
In any case, the next version of dcollections will support const to a  
certain degree (with the exception of iterable const ranges) when the next  
version of the compiler comes out.

Here is the commit which adds inout/const to dcollections:  
http://www.dsource.org/projects/dcollections/changeset/114

Be sure to use the latest version of dmd from git to try it out (if you're  
so inclined).

-Steve